Briefings in Bioinformatics, 2026, pp. 1–10 DOI here Published: Date added during production Paper
Plausibility-Driven Prioritization of Candidate Biomedical Annotations Emanuele Cavalleri ,1 Miad Alavinezhad ,1 Dario Malchiodi 1 and Marco Mesiti 1,∗ 1
Department of Computer Science, University of Milano, Milano, Italy ∗
Corresponding author. [email protected]
Abstract
arXiv:2607.20163v1 [q-bio.QM] 22 Jul 2026
The rapid growth of biomedical knowledge has made the validation of automatically generated biological annotations a major bottleneck in biomedical curation. While computational methods can rapidly produce large numbers of candidate annotations, determining which are biologically valid still requires costly expert review. Prioritizing these candidates before manual curation has therefore become a fundamental challenge. Machine learning techniques can support this process by exploiting biomedical knowledge graphs (bioKGs), which capture biological entities and their functional associations. In this work, we propose a framework that leverages bioKGs to estimate the plausibility of candidate annotations and guide expert curation. Starting from knowledge graph embeddings, we train relation-specific binary classifiers using a community-based negative sampling strategy to obtain reliable confidence estimates. We then introduce a family of plausibility measures that combine classifier confidence, classifier reliability, and the semantic context provided by alternative relationships involving the same pair of biological entities. Unlike conventional confidence estimation, the proposed approach explicitly accounts for multiple biologically meaningful relations that may coexist between the same entities. Experimental results on five large bioKGs demonstrate that the proposed negative sampling strategy consistently improves classifier robustness, increasing balanced accuracy by an average of 5.8%. Moreover, the plausibility measures outperform classifier confidence alone, enabling more effective prioritization of candidate annotations for expert review. Overall, our results show that the use of bioKGs improves the efficiency of AI-assisted biomedical curation while preserving expert control over the final annotation assessment. Keywords Plausibility of biomedical annotations, classification approach, biomedical annotation prioritization, AI assistant for biomedical curator
Graphical Abstract
Key Messages • • •
The main challenge in modern biomedical curation is not generating candidate annotations, but efficiently supporting expert validation as biomedical knowledge continues to grow. The contextual information encoded in bioKGs enables a more principled estimation of annotation plausibility than conventional confidencebased prediction. AI-driven plausibility estimation can substantially improve the efficiency and scalability of biomedical curation while preserving expert control over the final validation process.
Received: Date Month Year. Revised received: Date Month Year. Accepted: Date Month Year © The Author(s) 2026. Copyright and licence statements to be updated by the publisher during production.
2 Briefings in Bioinformatics, 2026, Volume XX, Issue x
Introduction
•
Biomedical research is generating new knowledge at an unprecedented pace, making the validation of biological annotations an increasingly critical challenge. More than 230,000 publications on non-coding RNAs have appeared since 2000, whereas only about 100–200 biocurators are responsible for transforming this continuously growing literature into structured representations [16]. Since only a small fraction of the available knowledge can be manually curated [19], computational methods operating on Biomedical Knowledge Graphs (bioKGs) [5] are needed to assist curators in prioritizing and validating candidate annotations while preserving expert oversight. BioKGs provide a unified semantic representation of biomedical entities and their relationships by integrating curated databases, experimental observations, and scientific literature. Representative examples include PrimeKG [12], PheKnowLator [3], RNAKG [10, 11], Hetionet [21], and OptimusKG [33]. In bioKGs, annotations are represented as triples (s, p, t), asserting that predicate p holds between entities s and t. For example, (miR-21, over-expressed-in, breast cancer) states that microRNA miR-21 is over-expressed in breast cancer. Candidate annotations may be generated by literature mining systems [15, 36], LLMs [7, 6, 19], high-throughput experiments [14], data integration pipelines [3, 10], or Knowledge Graph Embedding (KGE) methods [39]. Regardless of their origin, these annotations require expert validation before inclusion in curated bioKGs. Consequently, recent AI-assisted biocuration systems focus on supporting expert review rather than replacing human judgement [19]. Among the available techniques, KGEs are particularly suitable because they learn low-dimensional representations capturing the structural and semantic information encoded in bioKGs. Originally introduced for KG completion [13], these embeddings provide effective features for learning relation-specific classifiers that estimate the confidence of candidate annotations. However, confidence alone is insufficient because candidate annotations should also be evaluated in the context of alternative relationships involving the same entities. We therefore introduce the broader notion of plausibility, which combines classifier confidence with contextual evidence encoded in the bioKG to provide a stronger basis for prioritizing expert review. We propose a framework that exploits bioKGs to support expert curation through plausibility estimation. Starting from graph embeddings, we train relation-specific binary classifiers on homogeneous graph partitions using a community-based negative sampling strategy [9]. We then introduce plausibility measures that extend classifier confidence by incorporating classifier reliability and evidence provided by competing predicates between the same entity pair. The resulting scores distinguish highly plausible, implausible, and uncertain candidate annotations, enabling more effective prioritization for expert review. Experiments on five large bioKGs show the effectiveness of the proposed framework. community-based negative sampling improves balanced accuracy by 5.8% on average and yields higher confidence on unseen positive annotations. Moreover, the plausibility measures better discriminate target predicates from competing alternatives, improving candidate annotation prioritization while preserving expert control over the final validation process. Our main contributions are:
• • • •
a bioKG-based framework for estimating the plausibility of candidate biomedical annotations; a community-based negative sampling strategy that improves the robustness of relation-specific classifiers; three plausibility formulations combining classifier confidence, classifier reliability, and competing predicates; a set of evaluation metrics for assessing plausibility formulations; an extensive experimental validation on five large bioKGs.
Related Work KGE models [39] learn low-dimensional representations of entities and relations that preserve the structural and semantic information of a KG. These representations have proved effective for several biomedical tasks (e.g. gene–disease association prediction [29], drug interaction prediction [18], and drug repurposing [31]). Early methods (e.g. DeepWalk [30], node2vec [20]) exploit random walks for KGE, while more recent approaches learn embeddings for entities and relations relying on their types. Representative approaches include translation-based methods (e.g. TransE [1], TransH [35]), bilinear models (e.g. DistMult [37]), and complex-valued formulations (e.g. ComplEx [32], RotatE [24]). In this paper, we use KGEs only to derive latent representations that serve classifiers for confidence and plausibility assessment. Binary classifiers have been widely adopted to estimate candidate annotations [26] through confidence scores. Several approaches formulate triple validation as a supervised classification problem. For example, KGBoost [34] relies on gradient-boosted decision trees, KG-BERT [38] exploits transformer representations, and recent twostage architectures [22, 25] refine link prediction outputs through supervised classification. In this paper, we employ binary rather than multiclass classifiers because multiple biologically meaningful relations may exist between the same pair of entities. Binary models estimate the confidence of each predicate independently, without assuming mutual exclusivity. Negative sampling is essential for KG learning even if explicit negative triples are rarely available. Most existing methods generate artificial negatives during embedding learning, using strategies such as corruption-based, degree-aware, adversarial, dynamic, or influence-based sampling [2, 23, 28, 40]. A smaller body of work investigates negative sampling for downstream supervised classifiers [4], where the objective is to improve classification performance rather than embedding quality. Our work also targets downstream classification but exploits the heterogeneous semantics of bioKGs. By considering relation-specific communities [9], the proposed approach generates stronger negative annotations, improving the robustness of the confidence estimators.
Preliminary Concepts Biomedical Knowledge Graphs Schemas provide the semantic structure of bioKGs by defining entity types (e.g. genes, drugs, diseases, pathways) and relation types (e.g. treats, regulates, associated-with), ensuring semantic consistency. Definition 1 (KG Schema). Let NT be the set of node types and ET the set of edge types (predicates). A KG schema is a graph GS = ⟨NT , ES , ET ⟩, where ES = {(sT , p, tT ) | sT , tT ∈ NT , p ∈ ET } is the set of schema facts (denoted Facts(GS )).
Briefings in Bioinformatics, 2026, Volume XX, Issue x
(a) Knowledge Graph G
(b) Schema GS
3
(c) Partition of G by schema facts
Figure 1 Example of partitioning a KG G according to its schema GS .
A schema acts as a blueprint for the construction of KGs. Definition 2 (KG). Let GS = ⟨NT , ES , ET ⟩ be a KG schema. A Knowledge Graph conforming to GS is a triple G = ⟨N, E, ϕ⟩, where: N is the set of graph nodes (entities), ϕ a function assigning a type to each node, and E ⊆ N ×ET ×N is the set of typed edges, namely E = {(s, p, t)|ϕ(s), ϕ(t) ∈ NT , p ∈ ET , (ϕ(s), p, ϕ(t)) ∈ ES }. Given a KG G, we denote by Nodes(G) and Edges(G) its sets of nodes and edges. For a pair of entities s, t ∈ N , we define Facts(s, t) = {f ∈ Facts(GS )|f = (ϕ(s), p, ϕ(t))}, that is, the set of schema facts compatible with the types of s and t. For a schema fact f = (sT , p, tT ) ∈ Facts(GS ), we define the induced subgraph Gf as the subgraph of G containing all nodes of type sT or tT and all edges labeled by predicate p. Although the nodes of Gf may belong to two different types, all its edges share the same relation type. The collection of induced subgraphs associated with the schema facts forms a partition of Edges(G), namely S T f ∈Facts(GS ) Edges(Gf ) = E and f ∈Facts(GS ) Edges(Gf ) = ∅. Example 1 Figure 1a shows a KG describing relationships among RNA molecules, genes, diseases, and biological processes. Its schema (Figure 1b) allows multiple relation types between entities of the same type. Finally, Figure 1c illustrates the induced subgraphs obtained by partitioning the KG according to the schema facts in Facts(GS ).
Figure 2 Pipeline for training classifiers in a KG.
together with generated negative examples (N egative(fi )), from which the corresponding classifier learns to distinguish plausible from implausible annotations. Since classifier reliability depends not only on predictive performance but also on the amount of available training data, each schema fact fi is associated with a relevance weight wi combining the balanced accuracy bAcc(fi ) achieved on the validation set and the size of its training partition. wi is defined as the weighted harmonic mean of the normalized balanced accuracy bai and the normalized partition size si :
KG Embedding and Binary Classifiers A KG G is first embedded into a latent space using a KGE model, which captures both graph topology and relation types. The resulting edge embeddings are derived from the corresponding node embeddings (e.g. through concatenation or the Hadamard product). For each schema fact fi ∈ Facts(GS ), we train a relation-specific binary classifier Modelfi on the corresponding edge embeddings. This partitioning produces homogeneous training sets while preserving the heterogeneous semantics of the original bioKG. Given a candidate annotation (sN , p, tN ), Modelfi estimates its compatibility with the structural and semantic patterns observed in the corresponding partition Gfi . Negative examples are generated during training to improve classifier robustness (see next section). Figure 2 summarizes the training pipeline. Node and edge embeddings are first computed and then partitioned according to the schema facts. Each partition contains observed edges (P ositive(fi ))
wi = (1 + β 2 )
si bai , β 2 si + bai
where β > 1 emphasizes balanced accuracy, whereas β < 1 gives greater relevance to the partition size. In the formula, bai is defined as:
bAcc(fi ) . bai = Pn k=1 bAcc(fk ) while si is defined as: |Gf | si = Pn i . k=1 |Gfk |
4 Briefings in Bioinformatics, 2026, Volume XX, Issue x Method
Competition
Weight
Scope
Purpose
Pbase Pgain Pcombo Psoft
– ✓ ✓ ✓
– ✓ ✓ ✓
– strongest strongest top-k
Confidence estimation Maximize predicate separation Balance calibration and separation Multi-predicate competition
Table 1 Summary of the proposed plausibility formulations.
Gain Plausibility
Figure 3 Community-based negative sampling strategy.
The Negative Selection Strategy In [9], three negative sampling strategies were compared, and the community-based strategy achieved the highest balanced accuracy. The method exploits the community structure of the graph induced by a schema fact fi , computed through a graph clustering algorithm (e.g. Louvain [17]). Since nodes within the same community are more likely to be connected, negative examples are generated between nodes of different communities. Definition 3 (Community-based Strategy). Let fi = (sG , p, tG ) be a schema fact of a KG G, Gfi the induced graph, and {C1 , . . . , Cn } the communities identified by a graph clustering algorithm. A negative fact (sN , q, tN ) ∈ / Edges(Gfi ) is added to Gfi iff: (i) sN , tN ∈ Nodes(Gfi ), ϕ(sN ) = sG , ϕ(tN ) = tG , q ∈ ET ; (ii) ((sN , q, tN ) ∈ E ∧q ̸= p)∨((sN , q, tN ) ∈ / E ∧q = p); (iii) sN ∈ Ci , tN ∈ Cj , i ̸= j; (iv) |Negative(Gfi )| ≤ |Positive(Gfi )|. Figure 3 illustrates the strategy on the schema fact (Gene, associated-with, Disease). Negative examples (red) are generated between entities belonging to different communities.
Competition-Aware Plausibility Formulations Estimating the plausibility of a candidate annotation (sN , p, tN ) ∈ / Edges(G) means assessing how strongly the candidate is supported by the structural and semantic information encoded in the bioKG. The simplest solution is to rely on the confidence assigned by the binary classifier of the corresponding schema fact. Definition 4 (Base Plausibility). Let fi be the schema fact associated with (sN , p, tN ), and M odelfi the corresponding classifier. The Base Plausibility Pbase of (sN , p, tN ) is defined as: Pbase (sN , p, tN ) = M odelfi (sN , p, tN ). Although simple and effective, this formulation ignores two important aspects: (i) different predicates may provide alternative explanations for the same pair of entities; and (ii) classifiers differ in reliability according to both their validation performance and the amount of available training data. To address these limitations, we progressively introduce three competition-aware formulations that rely on the following base definition. Table 1 summarizes their main characteristics.
Inspired by learning-to-rank approaches [27], we introduce a competition-aware plausibility formulation based on the concept of relative gain, which measures the advantage of the target predicate over its strongest competitor. Since classifiers differ in reliability, each confidence score is weighted by the relevance of its schema fact, which combines classifier quality and the amount of available training data. The relative gain of schema fact fi is defined as ∆fi (sN , p, tN ) = M odelfi (sN , p, tN ) · wi −max{M odelfj (sN , q, tN ) · wj , 0} j̸=i
where the maximum is computed over all schema facts sharing the same source and target node types as fi , but associated with a different predicate q ̸= p. The constant 0 ensures that the formulation remains well defined when no competing predicate exists. Positive gain values indicate that the target predicate is preferred over its strongest competitor, whereas negative values indicate that an alternative predicate receives stronger support. Values close to zero identify cases in which competing predicates receive comparable support and therefore deserve further inspection. Definition 5 (Gain Plausibility) The Gain Plausibility of the candidate annotation (sN , p, tN ) for the schema fact fi is expressed as Pgain (sN , p, tN ) = σ(λ ∆fi (sN , p, tN )) , 1 where σ(z) = 1+exp(−z) is the sigmoid function and λ > 0 controls the sharpness of the transformation.
Combo Plausibility While Gain Plausibility compares the target predicate with its strongest competitor, this comparison may become overly conservative when multiple biologically meaningful predicates receive similarly high confidence scores. In such situations, penalizing the target predicate solely because a competing predicate is also plausible may underestimate its actual biological relevance. To alleviate this effect, we introduce Combo Plausibility, which combines the classifier confidence with the competition-aware information provided by Gain Plausibility. The resulting formulation preserves the good calibration of the Base formulation while exploiting the relative ranking among competing predicates. Definition 6 (Combo Plausibility) Let α ∈ [0, 1] be a mixing parameter. The Combo Plausibility of the candidate annotation t̄ = (sN , p, tN ) is defined as Pcombo (t̄) = α Pbase (t̄) + (1 − α) Pgain (t̄) where α controls the contribution of the classifier confidence and the competition-aware component.
Briefings in Bioinformatics, 2026, Volume XX, Issue x KG
|N |
|E|
|NT |
most repr. NT
|ET |
most repr. schema fact
miRNA-KG
99K
1.6M
Hetionet
47K
2.3M
27
BP (27K)
138
Disease HAS Phenotype (374K)
11
Gene (21K)
16
Genes participates BP (569K)
PKT-KG
560K
5.5M
23
Chemical (196K)
320
Disease HAS Phenotype (515K)
PrimeKG
129K
8.1M
10
BP (29K)
18
Drug SYN INT Drug (2.7M )
degreeAV G
OptimusKG 191K 21.8M 10 Gene (61K) 36 Disease associated-with Gene (9.7M ) Abbreviations: BP (Biological Process); SYN INT (synergistic-interaction); HAS (has-phenotype).
pathLenAV G
clusCoefAV G
32.62
5.22
0.11
95.69
3.60
0.11
19.71
4.57
0.05
125.29
4.86
0.08
228.95
4.01
0.07
5
Table 2 Datasets.
SoftMax Plausibility
Experimental Protocol
To capture a broader competitive context, where several alternative predicates can co-exist, we introduce SoftMax Plausibility, which compares the target predicate with the top-k competing predicates through a softmax normalization.
For each schema fact fi , the observed triples were first divided into two disjoint subsets. A random 10% was held out before training to build the blind positive test set BlPi , which is used exclusively for plausibility evaluation. Since bioKGs do not contain explicit negative facts, an equally sized set of implausible triples Ni was generated through the adopted negative sampling strategy. Neither BlPi nor Ni is used during training, validation, or model selection. The remaining 90% of observed triples were used to train the relationspecific classifiers (using 70-30% splitting to train/validate). For each schema fact, negative training examples were generated either through random sampling or through the proposed community-based strategy, depending on the experiment. This protocol separates classifier training from plausibility evaluation.
Definition 7 (SoftMax Plausibility). The SoftMax Plausibility of t̄ = (sN , p, tN ) according to fi is:
Psoft (t̄) = s
exp(λ·M odelfi (sN , p, tN )·wi ) P exp λ·M odelfj (sN , pj , tN )·wj
fj ∈SkN
,tN ∪{fi }
where SksN ,tN = arg maxfi ∈F act(sN ,tN ),|S|=k M odelfi (sN , pi , tN ) is the set of the topk schema facts with the highest scores, and λ > 0 controls the sharpness of the softmax normalization.
Metrics Metrics for evaluating classifiers
Experiments We evaluated the proposed framework on five large bioKGs to assess: (i) the effectiveness of the community-based negative sampling strategy; (ii) the impact of classifier and embedding choices; (iii) the proposed plausibility formulations; and (iv) the usefulness of plausibility scores in a real scenario. Node embeddings have dimension 32, edge embeddings are obtained via the Hadamard product, and RF and MLP classifiers are used. Hyperparameters were selected via nested cross-validation.
Datasets Table 2 summarizes the five bioKGs used in our evaluation, together with their main structural characteristics. The datasets cover complementary biomedical domains, including gene–disease associations, RNA interactions, drug–disease relationships, phenotypes, and biological processes, and range from 99K to 560K nodes and from 1.6M to 21.8M edges. The considered bioKGs exhibit typical properties of large biological networks, including small average path lengths, low clustering coefficients, and highly skewed degree distributions (details in the Supplementary Materials). To ensure meaningful classification tasks, we excluded schema facts containing fewer than 1000 observed triples or representing purely ontological assertions (e.g. gene,is-a,protein-coding gene). After filtering, we retained 12 schema facts for miRNA-KG, 21 for Hetionet, 20 for PKT-KG, 22 for PrimeKG, and 23 for OptimusKG. For partitions containing more than 1.5M edges, we randomly sampled 1.5M triples to keep the computational cost manageable while preserving representative training data.
Classifier performance is evaluated using standard metrics, including precision, recall, specificity, balanced accuracy, Matthews correlation coefficient, and the Fβ -score. In addition, we introduce the F̂β score, which jointly penalizes false positive and false negative rates: NR F̂β = (1 + β 2 ) β 2FFPNR·F . Unlike the traditional Fβ -score, which R+F P R balances precision and recall, F̂β directly measures the tradeoff between false assertions and missed positives. Throughout the experiments, we use β = 1, giving equal importance to the two error types. Hyperparameters are selected through nested cross-validation using F̂1 as the optimization objective. Details of the hyperparameter search are reported in the Supplementary Materials.
Metrics for evaluating plausibility A plausibility formulation should (i) distinguish unseen plausible triples from generated negatives and (ii) discriminate the target predicate from alternative predicates involving the same entity pair. Accordingly, we introduce two complementary families of metrics: calibration metrics and competition metrics. All metrics are first computed for each schema fact fi and then averaged to obtain a global evaluation over the entire KG.
Calibration metrics Calibration measures the ability of a plausibility formulation to assign high scores to plausible triples and low scores to implausible ones. The Positive Acceptance Rate (PAR) measures the average plausibility assigned to the blind test set:
P ARi =
X 1 P(s, p, t). |BlPi | (s,p,t)∈BlP i
6 Briefings in Bioinformatics, 2026, Volume XX, Issue x KG
Community
Random
Shared
miRNA-KG
147.3K
255.7K
5.0K (1.2%)
Hetionet
1.6M
2.2M
56.7K (1.5%)
PKT-KG
1.1M
1.7M
35.7K (1.3%)
PrimeKG
2.3M
4.4M
170.8K (2.6%)
OptimusKG
3.7M
6.1M
336.6K (3.5%)
Total
8.8M
14.8M
604.8K (2.6%)
Table 3 Overlaps of negative edges between the community-based strategy and random baseline.
The Best-Competitor Separation Rate is
Sepbest = i
Figure 4 Graphical interpretation of best-competitor separation rate. Green regions indicate triples for which the target and competitor scores fall on opposite sides of the threshold, contributing 1 to the metric; red regions indicate triples for which both scores fall on the same side, contributing 0.
Conversely, the Negative Rejection Rate (NRR) measures the average rejection of generated negatives: N RRi = 1 −
X 1 P(s, p, t). |Ni | (s,p,t)∈N i
Finally, the Calibration Rate combines the two quantities: Cali =
P ARi + N RRi . 2
High values of these metrics indicate that plausible triples receive high plausibility scores while negative ones are correctly rejected.
Competition metrics Calibration alone is insufficient when multiple biologically meaningful predicates may connect the same pair of entities. A plausibility formulation should also distinguish the target predicate from competing alternatives. To assess this property, we introduce three competition-oriented metrics. The Average Competitor Separation measures the average plausibility difference between the target predicate and all competing predicates associated with the same entity pair: Sepi =
X X 1 |P(s, p, t) − P(s, pj , t)| . |BlPi ||Ci | (s,p,t)∈BlP p ∈C i
j
i
Higher values indicate better separation of competing predicates. The previous metric equally weights all competing predicates. However, in practice the strongest competitor often represents the most informative alternative explanation for a candidate triple. Let P best (s, p, t) = max P(s, pj , t) pj ∈Ci
denote the plausibility assigned by the strongest competing predicate. We define the schema-fact-specific threshold bbest = i
P ARi + P ARibest . 2
1 |BlPi |
X
I (P(t̄) ≥ bbest ) ⊕ (P best (t̄) ≥ bbest ) . i i
t̄∈BlPi
It measures the fraction of triples for which the target predicate and its strongest competitor fall on opposite sides of the threshold. To evaluate whether the target predicate remains distinguishable even from the least plausible competing alternative, P worst is introduced to identify the minimal plausibility (bworst and Sepworst i i are defined accordingly). Figure 4 shows the interpretation of Sepbest . Each point represents a blind plausible triple according to the plausibility assigned by the target predicate and its strongest competitor. The threshold partitions the space into four regions. Triples lying in the Target/Competitor preferred regions contribute positively to the metric, whereas triples in the Rejected/Ambiguous regions indicate insufficient separation between competing predicates.
Selecting the Experimental Configuration Before evaluating plausibility, we compare the proposed communitybased negative sampling strategy with a random baseline and identify the experimental configuration adopted in the remainder of the paper. Table 3 reports the overlap between the negative triples generated by the two strategies. Across the five bioKGs, at most 3.5% of the generated negatives are shared, indicating that the communitybased strategy produces substantially different training examples rather than a small variation of random sampling. Consequently, any performance differences observed in the following experiments can be attributed to the quality of the generated negatives rather than to trivial overlaps. Moreover, negative generation is computationally inexpensive, requiring less than one minute per KG partition. To determine the training configuration, we evaluated different negative-to-positive ratios and two classification models (RF and MLP) on miRNA-KG and PKT-KG. As shown in Table 4a, both classifiers achieve their highest balanced accuracy using a 1:1 ratio between positive and negative examples. RF consistently provides slightly higher accuracy and lower variance than MLP and is therefore adopted in the remaining experiments. Since the proposed framework is embedding-agnostic, we also compared five heterogeneous KGE models. RotatE achieves the highest balanced accuracy, followed closely by DistMult and TransE (Table 4b). However, TransE requires only five minutes to compute the embeddings, whereas the remaining models require several hours to provide only marginal accuracy improvements. We therefore adopt TransE as the default embedding model in the remainder of the paper.
Briefings in Bioinformatics, 2026, Volume XX, Issue x
(a) Average balanced accuracy with standard deviation
(b) Balanced accuracy
7
(c) Execution time
Figure 5 Different perspectives of classification performance. (a) Average balanced accuracy with standard deviation; (b) Regression analysis on balanced accuracy. (c) Execution time as functions of edge cardinality (log-scale) for the considered negative sampling strategies.
Evaluating the Classification Approach Figure 5a reports the average balanced accuracy achieved by the RF classifiers across the five bioKGs using two negative sampling strategies. The community-based strategy consistently outperforms the random baseline, yielding an average improvement of 5.8%, with gains observed on all datasets. The largest improvement is obtained on OptimusKG (+7.5%), confirming the superiority of the community-based strategy. Figure 5b further analyzes the relationship between classifier performance and the size of the schema-fact partitions. Balanced accuracy generally increases with the number of positive triples available for training, indicating that larger partitions provide richer structural information for learning robust classifiers. Across the entire range of partition sizes, the community-based strategy consistently achieves higher performance than random sampling. Detailed results for each schema fact are reported in the Supplementary Materials. To assess whether the observed improvements are statistically significant, we applied both a chi-square test of independence and McNemar’s test. The analyses confirm that the communitybased strategy consistently produces different and significantly more accurate predictions on unseen plausible triples. In particular, community-based classifiers assign higher confidence scores than the random baseline in 92 out of 98 schema facts (93.9%), with an average increase of 10.1 percentage points. Complete statistical results are provided in the Supplementary Materials.
Neg-to-Pos ratio 1
Model
0.25
0.5
MLP RF
83.8 ± 2.0 84.4 ± 1.1
87.6 ± 1.4 88.0 ± 0.8
88.9 ± 1.1 89.4 ± 0.7
2
3
88.1 ± 1.2 88.0 ± 0.6
86.6 ± 1.0 84.8 ± 0.7
(a) Across negative-to-positive ratios.
B. Acc. Time
TransE
DistMult
RotatE
ComplEx
TransH
89.4 ± 0.7 5 min.
89.5 ± 0.9 969 min.
90.7 ± 0.7 976 min.
81.2 ± 1.2 1075 min.
87.3 ± 0.9 1592 min.
(b) Across embedding models using community-based negatives.
Table 4 Balanced accuracy (mean ± std) and embedding time. Best values are reported in bold.
Finally, Figure 5c evaluates scalability by relating training time to partition size. The proposed strategy exhibits execution times comparable to the random baseline while maintaining a more favorable trend on the largest partitions, showing that the improved predictive performance is achieved without compromising scalability.
Quantitative Evaluation of Plausibility Scores Table 5 compares the four proposed plausibility formulations using the global calibration and competition metrics. Unless otherwise stated, the parameter configuration providing the best validation performance is adopted; a sensitivity analysis is reported in the Supplementary Materials. For calibration, Pbase achieves the highest values for P AR, N RR, and Cal, indicating that the classifier confidence alone provides the best discrimination between plausible and generated negative triples. The competition-aware formulations really improve predicate discrimination. Specifically, Pgain achieves the highest values for Sep and Sepbest , confirming that comparing the target predicate with competing alternatives greatly enhances predicate-level separation. Among the proposed formulations, Pcombo provides the best overall compromise between calibration and competition-aware discrimination. Although it does not maximize every metric individually, it consistently achieves balanced performance across both evaluation dimensions, making it the most suitable formulation for prioritizing candidate annotations. Overall, the quantitative evaluation shows that competition-aware plausibility formulations provide substantially richer information than classifier confidence alone. While the base formulation remains the best calibrated, the proposed competition-aware formulations markedly improve predicate discrimination, with Pcombo providing the most balanced behaviour across the two evaluation dimensions.
Metric
Base
Gain
Combo
SoftMax
PAR NRR Cal Sep Sepbest Sepworst
80.4% (+10.2) 86.5% (+2.5) 83.4% (+6.4) 30.2% (+0.1) 51.2% (+1.1) 62.5% (-0.9)
73.2% (-1.0) 52.8% (+5.4) 63.0% (+2.2) 49.7% (+3.6) 99.3% (±0.0) 69.4% (+0.8)
76.9% (+4.6) 69.7% (+3.9) 73.3% (+4.3) 37.2% (+2.2) 84.9% (+6.7) 77.7% (+2.6)
63.0% (+3.7) 82.8% (+3.9) 72.9% (+3.8) 48.0% (+3.8) 94.3% (+1.2) 72.3% (+0.8)
Table 5 Global plausibility evaluation across all schema facts. Best value for each metric is highlighted in bold.
8 Briefings in Bioinformatics, 2026, Volume XX, Issue x
(a) Community-based negatives
(b) Random negatives
Figure 6 Distribution of Pcombo scores for the schema fact (Disease,associates,Gene) in Hetionet. The vertical dashed lines delimit the plausibility regions.
Qualitative Analysis of Plausibility Distributions To better understand the behaviour of the proposed plausibility formulations in realistic curation scenarios, we now inspect the plausibility distributions produced for a representative schema fact. The goal is to assess whether the proposed formulation distinguishes implausible triples, highly plausible annotations, and biologically meaningful competing predicates requiring expert inspection. Figure 6 illustrates the Pcombo distributions obtained for the schema fact (Disease,associates,Gene) in Hetionet. Green bars correspond to blind plausible triples, red bars to generated negatives, while the blue and orange curves represent the plausibility assigned to the same disease–gene pairs by the competing predicates downregulates and upregulates. Panels (a) and (b) compare classifiers trained using community-based and random negatives. Three observations emerge. First, community-based negatives produce a clearer separation between plausible and implausible triples. The overlap between the corresponding score distributions decreases from 49.8% to 26.6%, indicating that the classifier assigns more decisive plausibility estimates. Second, competing predicates are not simply rejected. Most competitor scores concentrate in the intermediate plausibility region, reflecting that alternative disease– gene relationships may remain biologically meaningful even when they do not correspond to the target predicate. Finally, the resulting score distribution naturally supports the curation workflow. Highly plausible triples can be prioritized for validation, implausible ones can be safely discarded, whereas intermediate scores identify candidate annotations deserving further expert inspection. Similar behaviour was consistently observed across the remaining schema facts and bioKGs. Community-based negatives systematically reduced the overlap between plausible and implausible triples while preserving meaningful plausibility scores for biologically related competing predicates. Additional qualitative examples are reported in the Supplementary Materials.
Case Study: Plausibility-Guided Biomedical Annotation To illustrate the practical usefulness of the proposed framework, we consider a biomedical curation scenario in which candidate annotations are automatically extracted from scientific articles and subsequently reviewed by a domain expert. The objective is to evaluate whether plausibility scores can effectively prioritize candidate annotations before manual validation.
Figure 7 Agreement between plausibility regions and expert curation.
The study considers 12 scientific articles describing functional and pathway annotations involving RNAs, genes, proteins, chemicals, and environmental exposures. Candidate triples were extracted using an LLM-based relation extraction framework [8]. Extracted entities and relations were normalized according to standard ontologies (e.g. HGNC, KEGG, MeSH, and OBO), enabling direct integration with the considered bioKGs. Overall, the dataset contains 27 entities, 8 schema facts, and 29 candidate annotations. Each annotation was evaluated using Pcombo and assessed by a biomedical expert. Figure 7 summarizes the agreement between plausibility regions and expert assessment. Overall, 13 annotations (44.8%) fall into the agreement regions, where both plausibility and expert judgement identify the annotation as either plausible or implausible. An additional 13 annotations are assigned to the uncertain region. Rather than representing incorrect predictions, these cases identify candidate annotations requiring manual inspection, which is precisely the role of the proposed framework in supporting biomedical curation. Only three annotations disagree with the expert assessment. Two involve the schema fact (protein,located-in,GO term), where biologically meaningful protein–process associations receive low plausibility because the predicate located-in is poorly supported by the structural evidence encoded in the bioKG. These cases suggest that plausibility estimation may also help identify schemalevel inconsistencies by highlighting relations that are semantically inappropriate for the extracted entities. Two disagreements arise from weak structural support for a relation in the KG, whereas the third corresponds to a borderline plausibility score close to the decision threshold. Rather than indicating a clear model failure, this example illustrates the intrinsic uncertainty of borderline candidate annotations and suggests that future versions of the framework could further exploit adaptive thresholds or curator feedback to refine plausibility evaluation.
Briefings in Bioinformatics, 2026, Volume XX, Issue x
9
Discussion and Conclusions
Funding
In this paper, we addressed the problem of estimating the plausibility of candidate biomedical annotations by exploiting the contextual knowledge encoded in bioKGs. We proposed a relationspecific classification framework based on graph embeddings and a community-based negative sampling strategy for constructing informative training examples. The framework is independent of the underlying graph embedding model and introduces three competition-aware plausibility formulations (Gain, SoftMax, and Combo) that extend conventional confidence estimation by incorporating classifier reliability and competing predicates. Experiments on five large bioKGs demonstrated the effectiveness of the proposed approach. Community-based negative sampling consistently improved classifier performance over random sampling, yielding more reliable plausibility estimates. Moreover, the proposed formulations better separated target predicates from competing alternatives, enabling a clearer distinction between highly plausible, implausible, and uncertain annotations requiring expert review. More broadly, our results support the use of bioKGs in combination with machine learning as a decision-support tool for biomedical curation. As the number of candidate annotations exponentially increases, plausibility estimation provides a principled mechanism for prioritizing expert review while preserving human oversight. The proposed framework also has some limitations that motivate future work. First, plausibility is inferred solely from the structural information encoded in the KG and does not explicitly model semantic relationships among predicates. Incorporating ontologybased knowledge could better distinguish complementary from mutually exclusive relations. Second, the quality of the plausibility estimates depends on the underlying graph representations and relation-specific classifiers. Future work will investigate alternative embedding methods, including graph neural networks, and the integration of textual evidence from the biomedical literature and LLMs. Although evaluated on bioKGs, the framework is general enough to support annotation validation in other heterogeneous KGs.
This work was supported by the “Data Science and Decision Science for Digital Health” grant under the PSR-2025 funding scheme of the University of Milano. Support to D.M. has been granted by the National Plan for NRRP Complementary Investments (PNC) in the call for the funding of research initiatives for technologies and innovative trajectories in the health–project n. PNC0000003– AdvaNced Technologies for Human–centrEd Medicine (project acronym: ANTHEM). Computational resources were provided by INDACO Core facility, which is a project of High Performance Computing at the University of Milan.
Data Availability
References 1.
2.
3.
4.
5.
6.
7.
Code and data (including the manually annotated dataset for the case study) for reproducing experiments are open-source and available at https://github.com/BioDataUniMI/PlausibilityKG and https: //doi.org/10.5281/zenodo.21359879. 8.
Competing interests There is NO Competing Interest. 9.
Ethics statement This study does not involve human participants, animal subjects, or identifiable personal data. All experiments were conducted using publicly available bioKGs. The proposed methodology is intended to support the prioritization of candidate biomedical annotations for expert review and does not substitute expert curation.
10.
Author contributions statement M.M. and E.C. conceived the work; E.C., D.M., and M.M. developed the plausibility measures; E.C. and M.A. developed the methods and conducted the experiments; M.M. and E.C. wrote the paper; all the authors validated the work.
11.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, et al. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. Ming Cai, Zhuolin Deng, and Chen Xiong. IF-NS: A New negative sampling framework for knowledge graph embedding using influence function. Know.-Based Syst., 315(C), April 2025. Tiffany J Callahan, Ignacio J Tripodi, Adrianne L Stefanski, et al. An open source knowledge graph ecosystem for the life sciences. Sci. Data, 11(1):363, April 2024. Luca Cappelletti, Lauren Rekerle, Tommaso Fontana, et al. Node-degree aware edge sampling mitigates inflated classification performance in biomedical random walk-based graph representation learning. Bioinformatics Advances, 4(1), January 2024. Alberto Cattaneo, Stephen Bonner, Thomas Martynec, et al. The role of graph topology in the performance of biomedical knowledge graph completion models. Bioinformatics, 41(10):btaf547, 10 2025. Harry Caufield, Carlo Kroll, Shawn T O’Neil, et al. CurateGPT: A flexible language-model assisted biocuration tool, 2024. J Harry Caufield, Harshad Hegde, Vincent Emonet, et al. Structured Prompt Interrogation and Recursive Extraction of Semantics (SPIRES): a method for populating knowledge bases using zero-shot learning. Bioinformatics, 40(3):btae104, 02 2024. E. Cavalleri, A. Rastegar Mojarad, J Harry Caufield, et al. Schema-Driven Structured Information Extraction from Biomedical Literature via Large Language Models. Technical report, University of Milano, 2026. Emanuele Cavalleri, Miad Alavinezhad, Marco Mesiti, et al. Better Negatives, Better Predictions: Negative Sample Selection Strategies for Enhancing Biomedical KG Edge Classification. In Proc. of the Int’l Workshop on Graphenhanced LLMs for trustwOrthy Web data management, 2026. Emanuele Cavalleri, Alberto Cabri, Mauricio Soto-Gomez, et al. An ontology-based knowledge graph for representing interactions involving RNA molecules. Scientific Data, 11(1):906, Aug 2024. Emanuele Cavalleri, Paolo Perlasca, and Marco Mesiti. RNA-KG v2.0: an RNA-centered Knowledge Graph with Properties. NAR Genomics and Bioinformatics, 8(1):lqaf194, 01 2026.
10 Briefings in Bioinformatics, 2026, Volume XX, Issue x 12.
13. 14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27. 28.
Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine. Scientific Data, 10(1), February 2023. Zhe Chen, Yuehan Wang, Bin Zhao, et al. Knowledge graph completion: A review. IEEE Access, 8:192435–192456, 2020. The ENCODE Project Consortium. Expanded encyclopaedias of dna elements in the human and mouse genomes. Nature, 583(7818):699–710, 2020. The Europe PMC Consortium. Europe PMC: a full-text literature database for the life sciences and platform for innovation. Nucleic Acids Research, 43(D1):D1042–D1048, nov 2015. Valérie de Crécy-lagard, Rocio Amorin de Hegedus, Cecilia Arighi, et al. A roadmap for the functional annotation of protein families: a community perspective. Database, 2022, 01 2022. Pasquale De Meo, Emilio Ferrara, Giacomo Fiumara, et al. Generalized Louvain method for community detection in large networks. In Int’l Conf. on Intelligent Systems Design and Applications, pages 88–93, 2011. Aryo Pradipta Gema, Dominik Grabarczyk, Wolf De Wulf, et al. Knowledge graph embeddings in the biomedical domain: are they useful? A look at link prediction, rule learning, and downstream polypharmacy tasks. Bioinformatics Advances, 4(1):vbae097, 01 2024. Andrew Green, Nancy Ontiveros-Palacios, Isaac Jandalala, et al. Goflowllm—curating mirna literature with large language models and flowcharts. Bioinformatics, 42(1):btaf683, 01 2026. Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. In Proc. of Int’l Conf. on Knowledge Discovery and Data Mining, page 855–864. ACM, 2016. Daniel Scott Himmelstein, Antoine Lizee, Christine Hessler, et al. Systematic integration of biomedical knowledge prioritizes drugs for repurposing. Elife, 6, September 2017. Yuki Iwamoto and Ken Kaneiwa. Predicting from a different perspective: A re-ranking model for inductive knowledge graph completion. In Proc.of 21st Pacific Rim Int’l Conf. on Artificial Intelligence, page 299–304. Springer-Verlag, 2024. Bhushan Kotnis and Vivi Nastase. Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs, 2018. Thanh Le, Nam Le, and Bac Le. Knowledge graph embedding by relational rotation and complex convolution for link prediction. Expert Syst. Appl., 214(C), March 2023. Muzhi Li, Cehao Yang, Chengjin Xu, et al. Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion. In Proc. of the Conf. of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4349–4363, 2025. Xinyu Liang, Guannan Si, Jianxin Li, et al. A survey of inductive knowledge graph completion. Neural Comput. Appl., 36(8):3837–3858, December 2023. Tie-Yan Liu. Learning to rank for information retrieval. Found. Trends Inf. Retr., 3(3):225–331, March 2009. Tiroshan Madushanka and Ryutaro Ichise. Negative Sampling in Knowledge Graph Representation Learning: A Review, 2024.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38. 39.
40.
Saee Paliwal, Alex de Giorgio, Daniel Neil, et al. Preclinical validation of therapeutic targets predicted by tensor factorization on heterogeneous graphs. Scientific Reports, 10(1):18250, Oct 2020. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. CoRR, abs/1403.6652, 2014. Basel Taweel, Anthony G. Marson, and Nasir Mirza. A systems medicine strategy to predict the efficacy of drugs for monogenic epilepsies. Epilepsia, 63(12):3125–3133, 2022. Théo Trouillon, Johannes Welbl, Sebastian Riedel, et al. Complex embeddings for simple link prediction. In Proc. of the Int’l Conf. on Machine Learning, page 2071–2080. JMLR.org, 2016. Lucas Vittor, Ayush Noori, Iñaki Arango, et al. OptimusKG: Unifying biomedical knowledge in a modern multimodal graph, 2026. Yun-Cheng Wang, Xiou Ge, Bin Wang, et al. KGBoost: A classification-based knowledge base completion method with negative sampling. Pattern Recognition Letters, 157:104– 111, 2022. Zhen Wang, Jianwen Zhang, Jianlin Feng, et al. Knowledge graph embedding by translating on hyperplanes. Proc. of the AAAI Conf. on Artificial Intelligence, 28(1), Jun. 2014. Chih-Hsuan Wei, Alexis Allot, Robert Leaman, et al. PubTator central: automated concept annotation for biomedical full text articles. Nucleic Acids Research, 47(W1):W587–W593, 07 2019. Bishan Yang, Wen-tau Yih, Xiaodong He, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In Int’l Conf. on Learning Representations, 2015. Liang Yao, Chengsheng Mao, and Yuan Luo. KG-BERT: BERT for Knowledge Graph Completion, 2019. Hai-Cheng Yi, Zhu-Hong You, De-Shuang Huang, et al. Graph representation learning in bioinformatics: trends, methods and applications. Briefings in Bioinformatics, 23(1):bbab340, 09 2021. Hyunsik Yoo, Yeon-Chang Lee, Kijung Shin, et al. Directed Network Embedding with Virtual Negative Edges. In Proc. of ACM Int’l Conf. on Web Search and Data Mining, page 1291–1299. ACM, 2022.
Plausibility-Driven Prioritization of Candidate Biomedical Annotations Supplementary Material Emanuele Cavalleri, Miad Alavinezhad, Dario Malchiodi, Marco Mesiti This Supplementary Material is organized as follows. Sections S1–S2 describe datasets and implementation details. Sections S3–S5 provide additional quantitative analyses, while Section S6 presents qualitative examples.
S1 Degree Distributions of bioKGs Figure S1 reports the complementary cumulative degree distributions of the considered bioKGs. Across all graphs, a small number of highly connected hub entities coexist with a large number of sparsely connected nodes, revealing strongly heterogeneous connectivity patterns. This structural organization is consistent with the presence of densely connected local regions and motivates the use of community-based negative sampling, which exploits graph communities to generate harder and more informative negative examples.
S2 Hyperparameter Configuration This appendix reports the hyperparameter grids adopted for the classification models considered in the evaluation. For RFs, we tuned the number of estimators and the maximum depth of individual trees. The number of estimators ranged in {100, 200, 300, 400, 500}, while max_depth ∈ {None, 10, 20, 30, 50}. For MLPs, we tuned the regularization strength α and Supp. Fig. S1. Degree distributions of bioKGs. the network architecture. The regularization parameter was selected from 13 logarithmically spaced values in the interval [10−5 , 10−1 ], while the hidden-layer configurations were chosen from {50, 100, 200}. Each MLP was trained with early stopping, a tolerance of 10−4 , and a maximum number of 20K iterations.
S3 Detailed Classification Performance Tables S1–S4 detail the classification results obtained for each KG. We report the best five and worst five schema facts in terms of balanced accuracy, together with the negative sampling strategy (Strategy), edge cardinality (Edges), precision, recall, specificity, F1 -score, Matthews correlation coefficient (MCC), and F̂1 -score. Results are reported as mean and standard deviation over the outer folds of the nested cross-validation procedure.
S4 Statistical Validation of the Classification Results S4.1 Comparison of prediction distributions The community-based and random negative sampling strategies are trained using different sets of generated negative triples. Consequently, the predictions produced by the two classifiers cannot be directly paired over the complete evaluation datasets because each model is exposed to a different collection of negative examples. To assess whether the two strategies exhibit different prediction behaviors, we applied Pearson’s chi-square test of independence to the binary predictions obtained over the blind positive triples BlPi and the corresponding generated negatives Ni for each schema fact. The null hypothesis assumes that the predictions produced by the two strategies are statistically independent. Rejecting this hypothesis indicates that the two sampling strategies induce significantly different prediction distributions. Across the 98 evaluated schema facts, the null hypothesis was rejected (p < 0.001) in 60 cases, demonstrating a strong statistical association between the predictions produced by the two training strategies.
1
Strategy
Schema Fact
Edges
Precision
Recall
Spec.
B. Acc.
F1
MCC
F̂1
Community Random Community Community Community
miRNA - in similarity relationship with - miRNA miRNA - in similarity relationship with - miRNA Gene - genetically interacts with - Gene miRNA - regulates activity of - Gene Gene - causes or contributes to condition - Disease
1.5K 1.5K 2.0K 36.6K 36.2K
0.990±0.004 0.985±0.012 0.967±0.014 0.972±0.002 0.956±0.003
0.978±0.003 0.977±0.014 0.976±0.022 0.975±0.001 0.955±0.002
0.989±0.004 0.987±0.011 0.965±0.015 0.950±0.004 0.952±0.003
0.984±0.003 0.982±0.008 0.970±0.013 0.962±0.002 0.954±0.002
0.984±0.002 0.981±0.008 0.971±0.014 0.974±0.001 0.956±0.001
0.966±0.005 0.964±0.016 0.942±0.027 0.927±0.002 0.907±0.002
0.014±0.003 0.014±0.006 0.024±0.013 0.033±0.001 0.046±0.001
Community Mean Random Mean
21.3K 0.924±0.007 0.944±0.010 0.886±0.012 0.915±0.008 0.933±0.006 0.839±0.013 0.063±0.008 21.3K 0.850±0.009 0.878±0.012 0.857±0.010 0.868±0.008 0.864±0.008 0.736±0.016 0.128±0.007
Random Random Random Community Random
36.2K 4.4K 36.6K 1.4K 1.4K
Gene - causes or contributes to condition - Disease miRNA - has function - GO miRNA - regulates activity of - Gene miRNA - is causal somatic mutation in - Disease miRNA - is causal somatic mutation in - Disease
0.860±0.003 0.792±0.012 0.761±0.008 0.772±0.009 0.655±0.015
0.840±0.004 0.829±0.015 0.829±0.005 0.938±0.023 0.747±0.021
0.876±0.003 0.804±0.014 0.766±0.010 0.657±0.023 0.645±0.023
0.858±0.003 0.817±0.010 0.797±0.006 0.797±0.016 0.696±0.014
0.850±0.003 0.810±0.011 0.794±0.005 0.847±0.008 0.698±0.014
0.718±0.005 0.633±0.020 0.594±0.011 0.631±0.019 0.393±0.028
0.139±0.003 0.182±0.010 0.198±0.005 0.104±0.033 0.294±0.015
Supp. Table S1. Detailed performance on miRNA-KG. Strategy
Schema Fact
Edges
Precision
Recall
Spec.
B. Acc.
F1
MCC
F̂1
Community Community Community Random Community
Anatomy - expresses - Gene Gene - participates - Cellular component Gene - covaries - Gene Anatomy - expresses - Gene Gene - interacts - Gene
526.4K 73.6K 61.7K 526.4K 147.2K
0.988±0.000 0.911±0.001 0.926±0.002 0.803±0.002 0.894±0.002
0.996±0.000 0.956±0.002 0.888±0.002 0.976±0.000 0.897±0.002
0.935±0.001 0.858±0.001 0.925±0.002 0.785±0.003 0.857±0.003
0.966±0.001 0.907±0.000 0.906±0.002 0.881±0.002 0.877±0.001
0.992±0.000 0.933±0.001 0.906±0.002 0.881±0.001 0.895±0.001
0.949±0.001 0.827±0.001 0.812±0.004 0.769±0.003 0.754±0.002
0.007±0.000 0.067±0.002 0.090±0.002 0.044±0.000 0.120±0.001
Community Mean Random Mean
107.1K 0.819±0.007 0.855±0.007 0.794±0.011 0.824±0.006 0.835±0.005 0.656±0.011 0.153±0.006 107.1K 0.734±0.006 0.765±0.009 0.744±0.008 0.754±0.006 0.746±0.006 0.512±0.011 0.228±0.007
Community Random Random Random Random
97.8K 7.7K 7.6K 97.8K 3.6K
Anatomy - upregulates - Gene Disease - upregulates - Gene Disease - downregulates - Gene Anatomy - upregulates - Gene Disease - localizes - Anatomy
0.622±0.018 0.572±0.006 0.561±0.005 0.596±0.002 0.622±0.024
0.780±0.004 0.808±0.014 0.808±0.011 0.610±0.004 0.520±0.028
0.489±0.043 0.456±0.010 0.432±0.006 0.628±0.003 0.715±0.033
0.634±0.019 0.632±0.008 0.620±0.007 0.619±0.002 0.617±0.015
0.692±0.010 0.670±0.008 0.662±0.006 0.603±0.003 0.566±0.018
0.281±0.037 0.280±0.019 0.257±0.015 0.238±0.004 0.240±0.032
0.307±0.004 0.284±0.015 0.287±0.012 0.381±0.002 0.356±0.025
Supp. Table S2. Detailed performance on Hetionet. Strategy
Schema Fact
Edges
Precision
Recall
Spec.
B. Acc.
F1
MCC
F̂1
Community Community Community Community Random
Gene - interacts with - Protein Gene - genetically interacts with - Gene Protein - molecularly interacts with - Protein Gene - causes or contributes to condition - Disease Gene - genetically interacts with - Gene
142.1K 2.0K 341.1K 36.2K 2.0K
0.988±0.001 0.974±0.008 0.968±0.001 0.965±0.003 0.947±0.017
0.983±0.001 0.975±0.008 0.985±0.001 0.955±0.003 0.960±0.013
0.974±0.002 0.972±0.008 0.948±0.001 0.962±0.004 0.952±0.016
0.978±0.001 0.974±0.006 0.967±0.001 0.958±0.003 0.956±0.009
0.985±0.000 0.974±0.002 0.976±0.001 0.960±0.003 0.954±0.009
0.954±0.001 0.947±0.004 0.938±0.001 0.917±0.005 0.912±0.018
0.021±0.001 0.024±0.003 0.024±0.001 0.041±0.003 0.041±0.009
Community Mean Random Mean
86.4K 0.863±0.006 0.900±0.006 0.846±0.008 0.873±0.006 0.879±0.005 0.750±0.010 0.108±0.005 86.4K 0.824±0.006 0.850±0.007 0.832±0.008 0.841±0.005 0.835±0.005 0.685±0.009 0.144±0.005
Community Community Community Random Random
2.5K 16.1K 10.8K 16.1K 10.8K
GO - positively regulates - GO Protein - located in - Anatomy Protein - located in - Cell Protein - located in - Anatomy Protein - located in - Cell
0.711±0.015 0.698±0.006 0.690±0.009 0.646±0.005 0.617±0.005
0.930±0.022 0.696±0.012 0.708±0.005 0.673±0.020 0.597±0.009
0.660±0.018 0.715±0.005 0.680±0.014 0.668±0.005 0.666±0.003
0.795±0.014 0.706±0.007 0.694±0.007 0.671±0.008 0.632±0.006
0.806±0.017 0.697±0.009 0.699±0.004 0.659±0.012 0.607±0.007
0.607±0.039 0.410±0.015 0.389±0.012 0.341±0.017 0.264±0.011
0.115±0.031 0.295±0.007 0.305±0.006 0.329±0.009 0.365±0.005
Supp. Table S3. Detailed performance on PKT-KG. Strategy
Schema Fact
Community Community Community Community Random
Disease - phenotype present - Effect and/or phenotype 150.3K 0.973±0.001 0.969±0.001 0.928±0.003 0.948±0.002 Gene and/or protein - interacts with - Molecular function 69.5K 0.944±0.001 0.941±0.002 0.917±0.001 0.929±0.002 Drug - enzyme - Gene and/or protein 5.3K 0.971±0.003 0.884±0.013 0.971±0.003 0.927±0.008 Cellular component - interacts with - Gene and/or protein 83.4K 0.922±0.002 0.940±0.002 0.890±0.004 0.915±0.002 Anatomy - expression present - Gene and/or protein 1.5M 0.889±0.001 0.916±0.001 0.897±0.001 0.907±0.001
Edges
Precision
Recall
Spec.
B. Acc.
F1
MCC
F̂1
0.971±0.001 0.942±0.002 0.925±0.008 0.931±0.001 0.902±0.000
0.893±0.003 0.856±0.004 0.855±0.015 0.834±0.003 0.812±0.000
0.043±0.001 0.070±0.002 0.046±0.004 0.077±0.002 0.092±0.000
Community Mean Random Mean
253.0K 0.874±0.007 0.876±0.010 0.824±0.010 0.850±0.006 0.873±0.006 0.715±0.011 0.113±0.006 253.0K 0.789±0.009 0.753±0.013 0.819±0.009 0.786±0.007 0.768±0.009 0.576±0.014 0.198±0.007
Community Random Community Random Random
19.9K 19.9K 1.2K 2.6K 1.2K
Anatomy - expression absent - Gene and/or protein Anatomy - expression absent - Gene and/or protein Disease - phenotype absent - Effect and/or phenotype Disease - off label use - Drug Disease - phenotype absent - Effect and/or phenotype
0.756±0.003 0.674±0.007 0.651±0.018 0.606±0.016 0.562±0.030
0.933±0.003 0.666±0.013 0.516±0.054 0.578±0.019 0.466±0.057
0.444±0.010 0.710±0.011 0.748±0.042 0.661±0.016 0.674±0.035
0.689±0.004 0.688±0.006 0.632±0.012 0.620±0.014 0.570±0.027
0.835±0.002 0.670±0.008 0.574±0.031 0.591±0.016 0.508±0.043
0.451±0.007 0.376±0.013 0.273±0.021 0.240±0.028 0.143±0.054
0.119±0.004 0.310±0.006 0.327±0.024 0.376±0.014 0.403±0.027
Supp. Table S4. Detailed performance on PrimeKG.
2/6
Strategy
Schema Fact
Edges
Precision
Recall
Spec.
B. Acc.
F1
MCC
F̂1
Community Community Community Community Community
Anatomy - EXPRESSION PRESENT - Gene Anatomy - EXPRESSION ABSENT - Gene Disease - ASSOCIATED WITH - Gene Cellular component - INTERACTS WITH - Gene Molecular function - INTERACTS WITH - Gene
6.6M 2.2M 9.7M 105.3K 90.9K
0.995±0.000 0.962±0.000 0.954±0.000 0.915±0.002 0.931±0.001
1.000±0.000 0.946±0.001 0.987±0.000 0.922±0.002 0.890±0.003
0.950±0.001 0.961±0.000 0.895±0.001 0.897±0.003 0.923±0.002
0.975±0.000 0.953±0.000 0.941±0.000 0.909±0.001 0.907±0.002
0.997±0.000 0.954±0.000 0.970±0.000 0.918±0.001 0.910±0.002
0.970±0.001 0.906±0.000 0.902±0.000 0.820±0.003 0.811±0.003
0.001±0.000 0.045±0.001 0.024±0.000 0.089±0.001 0.091±0.001
Community Mean Random Mean Community Community Random Random Random
941.4K 0.855±0.008 0.885±0.009 0.804±0.011 0.844±0.008 0.868±0.007 0.701±0.015 0.125±0.008 941.4K 0.757±0.008 0.759±0.013 0.779±0.009 0.769±0.007 0.756±0.009 0.541±0.014 0.217±0.007
Drug - SYNERGISTIC INTERACTION - Drug Drug - OFF LABEL USE - Disease Drug - ASSOCIATED WITH - Phenotype Drug - INDICATION - Phenotype Drug - OFF LABEL USE - Disease
1.3M 1.1K 3.7K 1.0K 1.1K
0.667±0.001 0.663±0.036 0.686±0.014 0.595±0.032 0.526±0.018
0.962±0.001 0.659±0.038 0.573±0.032 0.542±0.026 0.506±0.052
0.404±0.002 0.689±0.033 0.765±0.006 0.668±0.037 0.591±0.027
0.683±0.001 0.674±0.035 0.669±0.017 0.605±0.024 0.549±0.020
0.788±0.001 0.661±0.036 0.624±0.025 0.567±0.025 0.515±0.035
0.453±0.001 0.347±0.069 0.345±0.032 0.211±0.050 0.097±0.039
0.072±0.001 0.325±0.034 0.303±0.009 0.384±0.029 0.445±0.015
Supp. Table S5. Detailed performance on OptimusKG. Metric
k=2
PAR NRR Cal Sep Sepbest Sepworst
63.0% (+3.7) 82.8% (+3.9) 72.9% (+3.8) 48.0% (+3.8) 94.3% (+1.2) 72.3% (+0.8)
k=1
k=3
64.7% (+3.3) 62.9% (+3.7) 80.4% (+4.5) 83.1% (+3.9) 72.5% (+3.9) 73.0% (+3.8) 49.7% (+3.6) 47.7% (+3.9) 99.3% (±0.0) 93.8% (+1.3) 69.4% (+0.8) 72.6% (+0.5)
Metric
α = 0.5
PAR NRR Cal Sep Sepbest Sepworst
76.9% (+4.6) 69.7% (+3.9) 73.3% (+4.3) 37.2% (+2.2) 84.9% (+6.7) 77.7% (+2.6)
(a)
α = 0.25
α = 0.75
75.1% (+1.8) 78.8% (+7.4) 61.2% (+4.7) 78.1% (+3.2) 68.1% (+3.2) 78.5% (+5.3) 42.9% (+3.0) 32.7% (+1.2) 94.5% (+2.2) 67.0% (+4.6) 76.1% (+1.9) 70.7% (+0.6) (b)
Supp. Table S6. Evaluation of parameter settings for (a) Psoft and (b) Pcombo across all schema facts. Best value for each metric is highlighted in bold. However, the BlPi samples are the same for both strategies. We therefore applied McNemar’s test on the binary predictions obtained for BlPi to assess whether the two strategies differ in their predictions on the same plausible instances. In this case, a small p-value has a different meaning: it indicates that the disagreements are significantly unbalanced toward one strategy. The test reveals a significant asymmetry (p < 0.001 in 85 out of 98 schema facts). S4.2 Comparison of classifier confidence Since McNemar’s test only indicates that the discordant predictions are unbalanced, we further compared the median scores assigned by the two strategies on the same BlPi samples to identify the strategy providing higher confidence to true positives. For each schema fact, we computed the median classifier score over the corresponding blind test set. Median values were preferred over means because they are less affected by the presence of a small number of extremely confident predictions. The community-based strategy assigns higher median confidence than the random baseline for 92 out of the 98 schema facts (93.9%). The average improvement is 10.13 percentage points, while the maximum observed increase reaches 31.80 points for the schema fact (Anatomy,expression-present,Gene) in OptimusKG. Moreover, the improvement exceeds five percentage points for 64 schema facts (65.3%) and ten percentage points for 45 schema facts (45.9%). These results indicate that community-based negative sampling systematically increases the confidence assigned to previously unseen plausible facts.
S5 Additional Quantitative Evaluation of Plausibility S5.1 Sensitivity to Parameter Selection The plausibility formulations introduce a small number of parameters controlling the contribution of classifier reliability and competition-aware information. To assess the robustness of the proposed approach, we evaluated different parameter configurations. For the Gain and SoftMax formulations, the scaling parameter λ was varied between 1 and 20. Small values produce smoother plausibility distributions, whereas larger values increasingly emphasize the differences between competing predicates. Performance remains stable, with λ = 10 providing the best compromise between calibration and predicate separation. For the Combo formulation, we evaluated the mixing parameter α between 0 and 1. Intermediate values provide the best trade-off, with α = 0.5 yielding the highest average performance across the proposed plausibility metrics. For the SoftMax formulation, we also investigated the number k of competing predicates included in the normalization. Small values focus on the strongest alternatives, whereas larger values incorporate increasingly weaker competitors. We 3/6
(a) Community-based negatives
(b) Random negatives
Supp. Fig. S2. Distribution of Pcombo scores for the schema fact (Disease,upregulates,Gene) in Hetionet. observed limited improvements beyond k = 2, indicating that the strongest competitors already capture most of the relevant competition among predicates. Table S6 reports additional parameter configurations for Psoft and Pcombo . Values are percentages; values in parentheses report the improvement, in percentage points, obtained by using community-based negatives instead of the random baseline. S5.2 Impact of Community-Based Negative Sampling The main paper reports the aggregate improvements obtained by replacing random negatives with the proposed community-based strategy. Here we provide a more detailed analysis. Across the 24 formulation–metric combinations considered in Table 5, community-based negatives improve performance in 22 cases, with an average gain of approximately three percentage points. The largest improvement is observed for the Base formulation on the Positive Acceptance Rate, where the increase exceeds ten percentage points. Since the Base formulation directly reflects the confidence assigned by the underlying classifier, this result confirms that community-aware negatives primarily improve the classifier’s ability to recognize previously unseen plausible triples. Competition-aware formulations also consistently benefit from the improved classifiers. Although the gains are smaller than those observed for the Base formulation, they are remarkably stable across the separation metrics, demonstrating that better classifier calibration naturally translates into more informative competition-aware plausibility estimates. S5.3 Analysis of Plausibility Score Polarization Besides calibration and competition-aware discrimination, we also investigated how the different plausibility formulations distribute scores over the interval [0, 1]. For each schema fact, we considered the plausibility scores assigned to blind plausible triples BlPi and generated negatives Ni , obtaining the empirical score distribution X. We quantify score polarization through the normalized dispersion index Var(X) D = Mean(X)(1−Mean(X)) . The denominator corresponds to the maximum variance attainable by a distribution on [0, 1] having the same mean. Consequently, D assumes values in [0, 1], where larger values indicate increasingly polarized score distributions. A continuous uniform distribution on [0, 1] has a dispersion index D = 1/3. Accordingly, we consider values exceeding this threshold as evidence of score polarization. Using community-based negatives, the Combo formulation satisfies this criterion in 74 out of 98 schema facts (75.5%), compared with 64 (65.3%) when random negatives are adopted. The same behavior is observed for all formulations. Base satisfies the criterion in 90 schema facts using community-based negatives, compared with 79 using random negatives. Gain increases from 59 to 66 schema facts, whereas SoftMax increases from 58 to 65. The difference becomes even more evident for highly polarized distributions. Considering all formulations together, the stricter criterion D > 2/3 is satisfied in 132 cases using community-based negatives, but in only 10 cases using random negatives. Similarly, D > 0.8 is observed in 63 cases with community-based negatives, compared with only three cases for random sampling.
S6 Additional Qualitative Analysis of Plausibility Distributions The quantitative evaluation presented in the main manuscript demonstrates the effectiveness of the proposed plausibility formulations through aggregate calibration and competition metrics. In this section, we provide additional qualitative examples illustrating how Pcombo distributions support the interpretation of candidate annotations under different biological contexts.
4/6
(a) Community-based negatives
(b) Random negatives
Supp. Fig. S3. Distribution of Pcombo scores for the schema fact (Gene,interacts,Gene) in Hetionet.
(a) Community-based negatives
(b) Random negatives
Supp. Fig. S4. Distribution of Pcombo scores for the schema fact (Gene,covaries,Gene) in Hetionet. In all figures, green histograms correspond to blind plausible triples (BlPi ), whereas red histograms correspond to generated implausible triples (Ni ). The blue and orange curves represent the plausibility assigned to competing predicates involving the same entity pair. Vertical dashed lines delimit the three plausibility regions introduced in the main paper, namely implausible, uncertain, and plausible. S6.1 Disease–Gene interactions in Hetionet Figure S2 reports the plausibility distributions obtained for the schema fact (Disease,upregulates,Gene). Similarly to the example discussed in the main paper, community-based negative sampling produces substantially clearer score distributions than the random baseline. The overlap between plausible and implausible triples decreases from 65.1% to 22.3%, allowing the proposed plausibility formulation to better distinguish highly plausible annotations from weakly supported candidates. An interesting observation concerns the competing predicate associates, which consistently receives relatively high plausibility scores. Rather than indicating an incorrect prediction, this behavior reflects the biological relationship between the predicates. Disease–gene associations represent a broader semantic concept that naturally includes more specific regulatory mechanisms such as gene upregulation. Consequently, high plausibility assigned to associates should be interpreted as evidence supporting a general biological relationship, while the more specific mechanistic annotation requires further expert validation. S6.2 Gene–Gene interactions in Hetionet. Figure S3 reports the distributions for the schema fact (Gene,interacts,Gene). In both strategies, plausible and implausible triples are well separated, but community-based negatives further reduce the overlap between the two distributions, from 12.6% with random negatives to 5.4%. The competing predicates regulates and covaries receive intermediate-to-high plausibility scores, which is coherent with the fact that they represent biologically related, but more specific or indirect, gene–gene relations. This behavior confirms that our approach does not simply reject competing predicates, but places them in regions where expert inspection may be useful. Figure S4 shows the complementary case (Gene,covaries,Gene). Community-based negatives lead to a clearer separation between plausible and implausible triples, reducing their overlap from 18.9% to 7.2%. The competitor regulates is mostly shifted toward lower or intermediate plausibility scores, while interacts receives broader intermediate-to-high scores. This is
5/6
(a) Community-based negatives
(b) Random negatives
Supp. Fig. S5. Distribution of Pcombo scores for the schema fact (miRNA,causes or contributes to condition,Disease) in miRNA-KG.
(a) Community-based negatives
(b) Random negatives
Supp. Fig. S6. Distribution of Pcombo scores for the schema fact (miRNA,under-expressed in,Disease) in miRNA-KG. consistent with the fact that gene co-variation can be compatible with interaction-like signals, but does not necessarily imply direct regulation. S6.3 miRNA–Disease interactions in miRNA-KG. Figure S5 reports the distributions for (miRNA,causes or contributes to condition,Disease). The two strategies both produce a strong separation between plausible and implausible triples, but community-based negatives slightly reduce the overlap between the two distributions, from 7.7% to 5.0%. Competing predicates receive distinct score profiles. For example, the predicate over-expressed in receives intermediate plausibility scores, suggesting that altered miRNA expression may provide evidence for a disease-related association without necessarily establishing the broader causes or contributes to condition relation. Figure S6 shows the case of (miRNA,under-expressed in,Disease). Here, community-based negatives assign a larger fraction of triples to the plausible region (29.3% versus 25.3%) and a smaller fraction to the uncertain region (45.7% versus 49.1%). The competitor curves remain biologically interpretable: over-expressed in and related disease–miRNA predicates are not necessarily unrelated alternatives. Indeed, under-expression and over-expression relations may coexist in the bioKG for the same miRNA–disease pair depending on the specific experimental condition, tissue, or biological context considered. For this reason, intermediate scores should not be interpreted as automatic errors, but as cases requiring expert assessment rather than direct rejection.
6/6