NeurOWL: An LLM-Based Neural-symbolic Framework for Incomplete OWL Ontology Reasoning Hui Yang1 , Jiaoyan Chen1 , Yiping Song1 , Renate Schmidt1 , and Wen Zhang2
arXiv:2607.15776v1 [cs.AI] 17 Jul 2026
1 The University of Manchester {hui.yang-2, jiaoyan.chen, renate.schmidt}@manchester.ac.uk, [email protected] 2 Zhejiang University [email protected]
Abstract. OWL ontologies provide a formal knowledge representation framework that enables semantic reasoning, and have been widely adopted across domains such as healthcare and bioinformatics. In practice, however, real-world ontologies are often incomplete, which pose challenges for reasoning. In this work, we focus on a fundamental subsumption reasoning problem: given an incomplete ontology and a candidate (nonentailed) subsumption, determine whether the subsumption is semantically plausible and, if so, providing a logically sound explanation containing potential missing axioms. This task unifies subsumption verification with ontology abduction, and generalizes the latter by removing the need for a predefined candidate set of missing axioms. To address this subsumption reasoning problem, we propose NeurOWL, an end-to-end neuro-symbolic framework that jointly performs verification and abduction, leveraging both formally defined semantics and textual semantics through Large Language Models and ontology embeddings. We evaluate NeurOWL on real-world ontologies across multiple domains, demonstrating strong and robust performance across different domains.
1
Introduction
Ontologies expressed in the Web Ontology Language (OWL) provide a formal framework for representing structured knowledge and enable automated reasoning through Description Logics (DLs) [3]. Owing to their precise and shareable representation of machine-interpretable semantics, OWL ontologies are widely used in knowledge-intensive domains such as biomedicine [2] and healthcare [7]. A fundamental reasoning task in ontology engineering is subsumption reasoning: determining whether a subsumption relationship holds between two concepts and providing explanations for such entailments. For complete and consistent ontologies, subsumption can be determined by standard reasoners [17, 24], and explanations can be derived as justifications (i.e., minimal sets of axioms sufficient for the entailment) [1,25,32,42]. In practice, however, real-world ontologies are often incomplete due to different reasons like evolving domain knowledge
2
Hui and Jiaoyan, et al.
and the high cost of expert curation. As a result, some valid subsumptions may not be entailed because some required axioms are missing. Abductive reasoning has been proposed as a natural solution to this problem [6, 11, 13–16, 18–20, 26–28, 33, 37]. In this work, we concentrate on TBox abduction that aims to find a set of hypotheses (missing axioms) H such that, when added to an incomplete ontology O′ , the target subsumption α becomes entailed (i.e., O′ ∪ H |= α). Despite their promise, existing approaches exhibit several limitations: (1) they typically rely on a predefined hypothesis space, restricting candidate axioms to a fixed vocabulary [19, 28]; (2) they often assume that the target subsumption is valid, preventing their applicability in more general settings where the validity of α is itself uncertain. Recently, there has been growing interest in using large language models (LLMs) for related ontology reasoning and construction tasks such as subsumption prediction and concept placement [5, 8, 21, 22, 31, 34, 36, 39, 43]; however, these works generally lack explanatory capabilities. To address these challenges, we propose NeurOWL, an end-to-end neuro-symbolic framework for explainable subsumption reasoning over incomplete OWL ontologies. By combing textual information and formal semantics through LLMs and ontology embeddings, NeurOWL removes the need for a predefined hypothesis space and supports both positive and negative entailment scenarios. Moreover, it operates in both training-free and fine-tuned settings, making it adaptable across domains. The core idea of NeurOWL is to uncover missing knowledge by identifying intermediate concepts that connect related notions, as illustrated in Figure 1. Consider a simple example (see Example 2 for the full version) with the ontology O1′ : {PersianCat ⊑ PetCat, Cat ⊑ Mammal, Mammal ⊑ Animal ⊓ ∃produces.Milk}. Intuitively, one expects the subsumption PersianCat ⊑ Animal to hold, but it cannot be derived from O1′ due to incompleteness. To address this, NeurOWL searches for intermediate concepts C such that PersianCat ⊑ C ⊑ Animal, which can be used to construct missing axioms and explanations. Depending on the scenario, part or all of the connections may be missing (cf. Stages 2a/2b and 3a in Figure 1). For example, NeurOWL may select C = Cat and construct the missing axiom PetCat ⊑ Cat, while the other part, Cat ⊑ Animal, is entailed by O1′ and is therefore not missing). Then, the explanation consist of the missing axioms and the axioms that entail Cat ⊑ Animal. Candidate intermediate concepts are identified using ontology embeddings [39] and LLMs, which capture semantic similarity and verify likely missing connections. If no suitable intermediate concept is found, NeurOWL treats the target subsumption as a simple, indivisible axiom, proposes it directly, and validates it using an LLM (Stage 3b in Figure 1). We evaluate NeurOWL on multiple real-world ontologies across diverse domains. Experimental results demonstrate that our approach achieves strong perfor-
Title Suppressed Due to Excessive Length
3
Input: A ⊑ B ? 1. Reasoner over O′
True proof justification JB ′ ⊑B
proved
no 2b. Upward
2a. Downward B O′ |= B ′ ⊑ B B′ A ⊑ B ′ missing? A
3a. Bidirectional
B fail
A′ ⊑ B missing? A′ O′ |= A ⊑ A′ A
True {A ⊑ B ′ } ∪ JB ′ ⊑B or {A′ ⊑ B}∪JA⊑A′
no 3b. Direct Check
B
B
C ⊑ B missing? fail
C
found
A ⊑ C missing?
A ⊑ B missing? A
A
found
True {A ⊑ C, C ⊑ B} or {A ⊑ B}
no
False
Fig. 1: Overview of NeurOWL: Red axioms denote potentially missing axioms.
mance in concept subsumption reasoning while also generating meaningful explanations for incomplete input ontologies. Our results shows that, on datasets constructed from the real-world ontologies FoodOn [12] and Snomed CT [7], NeurOWL achieves high performance in checking subsumption plausibility, with F1 scores of up to 97. They also demonstrate strong accuracy in explanations generated, as reflected in the correctness of predicted missing axioms, achieving X-F1 and X-F1* scores of up to 89.3 and 79.3, respectively. Further details are provided in Section 6. The main contributions of this work are: – We introduce a generalized TBox abduction task that does not require a predefined candidate set of missing axioms and handles cases where the given subsumption is incorrect. – We propose NeurOWL, an end-to-end neuro-symbolic framework for reasoning over incomplete ontologies, utilising ontology embedding and LLM. – We empirically evaluate our approach on real-world ontologies, demonstrating its effectiveness and robustness under varying degrees of incompleteness.
2
Related Work
Abductive Reasoning on ontologies can be divided into several categories, including TBox abduction [14,19,37], ABox abduction [11,13,15,18,20,26,27,33],
4
Hui and Jiaoyan, et al.
knowledge-base abduction [16,28], and concept abduction [6]. These categories are distinguished by the types of the missing elements: rules (TBox axioms), facts (ABox axioms), a combination of both, and concepts, respectively. In this work, we focus on TBox abduction, where the missing elements are TBox axioms, as introduced in Section 3. Existing approaches to TBox abduction rely on different assumptions regarding the subsumption of a predefined set of candidate axioms. For example, [37] assumes a human-verified selection defined by an oracle function provided by domain experts; [14] relies on a set of plausible patterns; and [19, 28] consider only axioms formulated over a given set of concepts and roles, known as signatures. These assumptions limit the practicality of such methods in real-world applications. Moreover, these approaches focus on logical computation and largely ignore the use of informal meta information like textual labels which also contain important semantics. Ontology Embeddings aim to encode ontology entities (i.e., concepts, roles, and instances) as numerical vectors while preserving their structural and semantic properties. These embeddings support several downstream machine learning and data mining tasks including concept subsumption prediction [10]. Existing approaches can be broadly divided into two categories. (1) Geometric modelbased methods [23, 29, 38, 40] represent ontology entities as geometric objects, translating description logic (DL) operators into geometric operations. For example, concept subsumption and intersection are modeled as region inclusion and intersection, respectively. (2) Language-model-based methods [9, 30, 35] leverage both formal knowledge and textual information to capture both linguistic and semantic features in the embedding space. More recently introduced, HiT [22] combines language models with hierarchical embeddings in hyperbolic space to represent taxonomies and support direct subsumption inference between two arbitrary concept labels. Building on this, OnT [39] extends the approach to EL ontologies by introducing role embeddings as rotations and additional loss functions to encode logical properties. In this work, we adopt OnT as our embedding method. LLMs for Ontology Subsumption have been investigated in many tasks such as subsumption prediction or inference [8, 21, 22, 39, 43], taxonomy completion [34], axiom learning [5, 31, 43], and the intrinsic knowledge of hierarchical structures in LLMs [36]. However, to the best of our knowledge, few studies have specifically explored the use of LLMs for assessing the semantic plausibility of subsumptions. [44] is a related approach but assuming additional verification by human experts. Moreover, most of these LLM-based works, as well as the above introduced ontology embedding-based works, focus on the correctness of the subsumption, and little attention has been devoted to explanations. [41] investigates explanation generation and considers incomplete settings, but assume the correctness of the given subsumption and focus on evaluating the capabilities of LLMs instead of developing an end-to-end reasoner.
Title Suppressed Due to Excessive Length
3
5
Preliminaries
Ontologies provide a formal framework for representing structured knowledge through collections of logical statements, known as axioms. These axioms describe relationships between concepts (unary predicates) and roles (binary predicates). Here, we take EL ontologies as an example, which is a lightweight yet expressive ontology that has been successfully employed in areas such as life sciences and semantic data integration [4]. Let NC = {A, B, . . .}, NR = {r, s, . . .} denote two mutually disjoint sets of concept names, role names, respectively. The set of EL-concepts is inductively defined as follows: C ::= ⊤ | A | C ⊓ D | ∃r.C, where A ∈ NC , r ∈ NR , and C, D are EL-concepts. In this work, we concentrated on TBox axioms, which also called General Concept Inclusions (GCIs) axioms of the form C ⊑ D, where both C and D are EL-concepts. Example 1. Given the concept names PersianCat, PetCat, Cat, Mammal, Animal, Milk and the role produces, we may define a simple ontology O1′ as follows: {PersianCat ⊑ PetCat, Cat ⊑ Mammal, Mammal ⊑ Animal ⊓ ∃ produces.Milk}. Logical reasoning over an ontology O enables the derivation of implicit knowledge from the stated axioms. The entailment relation “|=” is formally defined using interpretations and models (e.g., from A ⊑ B, B ⊑ C ∈ O, one can infer A ⊑ C, written as O |= A ⊑ C. See [4] for more details). To understand why an entailment holds, one can examine its justifications, which are minimal sets of axioms from which it follows [32]. Definition 1 (Justification). Let O be an ontology such that O |= A ⊑ B. A justification for A ⊑ B is a subset JA⊑B ⊆ O that is minimal with respect to set inclusion and satisfies JA⊑B |= A ⊑ B.
4
The NeurOWL System
The task we wish to solve is the plausibility of a given concept inclusion α = A ⊑ B being implied by a possibly incomplete ontology O′ , and return a set E of axioms as a possible explanation, i.e., We aim to determine the plausibility of given concept subsumptions over an O′ , E |= α. To ensure justifications rely primarily on the stated axioms in O′ we limit the approach to return at most two missing Concept Inclusion axioms, if possible. Without loss of generality, we focus on subsumptions of the form A ⊑ B between atomic concepts A, B. This setting generalizes naturally to subsumptions between complex concepts, C ⊑ D, since a reasoner can reduce complex expressions by introducing fresh concept names (e.g., A ≡ C and B ≡ D). Moreover, embeddings can be applied on complex concepts through verbalizations, as demonstrated in [39].
6
Hui and Jiaoyan, et al.
4.1
Main Structure
The overall NeurOWL framework is illustrated in Figure 1. It integrates logical structure with neural verification to predict missing axioms and consists of three stages: – Stage 1. Logical Checking: Determine whether the input query can be logically entailed by O′ using a standard ontology reasoner. – Stage 2. Logical Bridging: Identify potentially missing axioms using candidate concepts induced by the logical structure of O′ , combined with LLMbased validation. – Stage 3. Bridging by Embedding: Generate candidates directly based on semantic similarity using embeddings when the previous stages fail. In Stage 1 (Logical Checking), we use a standard description logic reasoner to test whether O′ |= A ⊑ B holds. If so, the system returns True with a justification derived from O′ . Otherwise, NeurOWL assumes some axioms are missing and proceeds to subsequent stages. In Stage 2 (Logical Bridging), we attempts to identify missing bridge concepts using the logical information in O′ . There are two cases: – Stage 2a: Downward. Consider each child B ′ of B in O′ (i.e., O′ |= B ′ ⊑ B) as candidate for a bridge concepts. Then, A ⊑ B ′ is treated as a candidate missing axiom and will be verified by the LLM. – Stage 2b: Upward. Consider each parent A′ of A in O′ (i.e., O′ |= A ⊑ A′ ) as candidate, and use the LLM to validate the corresponding potential missing axiom A′ ⊑ B. Each missing axiom candidate is then verified by an LLM using prompt learning. If a valid bridge is found, the system returns True along with the constructed justification. For example, if A ⊑ B ′ is validated, the justification is: {A ⊑ B ′ } ∪ JB ′ ⊑B , Note that LLMs may produce redundant answers that can be pruned directly. For instance, if B1′ and B2′ are selected in Stage 2a with O′ |= B1′ ⊑ B2′ , it suffices to retain B1′ with missing axiom A ⊑ B1′ , since A ⊑ B2′ follows from O′ ∪ {A ⊑ B1′ }. Example 2 provides a more realistic instance. In implementation, instead of directly calling LLM to check each potential missing axiom, NeurOWL first uses an ontology embedding model (e.g., OnT [39]) to retrieve the top-k candidates, which are ranked by the subsumption score defined by the embedding model. This not only reduces the cost and computation time, but also utilises more semantics embedded. In Stage 3 (Bridging by Embeddings), if no suitable candidates are found in the previous stage, we search for missing axioms based purely on semantic information.
Title Suppressed Due to Excessive Length
7
– Stage 3a (bidirectional search): We first perform a bidirectional search (Stage 3a in Figure 1) for an intermediate concept C such that A ⊑ C ⊑ B, where C is, by default, drawn from all atomic concepts in O′ (excluding A and B), but can be customized based on user interest (e.g., existential restriction concepts of the form ∃r.B1 ). We then select the top-k candidates and verify them using the LLM. Each candidate is ranked by the scoring function s̄(C) =
1 s(A ⊑ C) + s(C ⊑ B) . 2
If a valid C is identified, the system returns True with the justification {A ⊑ C, C ⊑ B}. Otherwise, the subsumption A ⊑ B is treated as an elementary assertion (i.e., requiring no further decomposition) and as a potential missing axiom itself. – Stage 3b (Direct check): Then, NeurOWL uses LLM to check the correctness of the subsumption A ⊑ B directly in Stage 3b, as shown in Figure 1. If the LLM judges A ⊑ B to be correct, NeurOWL returns True along with the subsumption itself as the missing axiom and explanation. Otherwise, NeurOWL indicating that the input subsumption does not plausibly hold and returns False. Example 2 (Continuation of Example 1). The ontology O1′ given in Example 1 is incomplete and miss the axiom: PetCat ⊑ Cat. Thus can not entail the following subsumption: PetCat ⊑ Animal. If we input this subsumption into NeurOWL, than we have: – Stage 1: The reasoner can not entail the given subsumption due to the incompleteness of O1′ . – Stage 2a (Downward): The system explores subclasses of Animal; in this simple case, only Mammal and Cat are considered, so no filtering step is required. This yields two candidate missing axioms: PetCat ⊑ Cat, PetCat ⊑ Mammal. A LLM is then used to select the axioms that are likely to be valid. If the LLM does not select any candidates, the process proceeds to the next stage. – Return: Suppose the LLM identifies both candidate axioms as valid. In this case, NeurOWL concludes that the queried subsumption holds, yielding two outputs. Note that PetCat ⊑ Mammal can be ignored, as it is derivable from O1′ together with PetCat ⊑ Cat. Finally, we output the missing axiom. PetCat ⊑ Cat, and the corresponding explanation: {PetCat ⊑ Cat} ∪ {Cat ⊑ Mammal, Mammal ⊑ Animal}, where the latter forms a justification for O1′ |= PetCat ⊑ Animal.
8
Hui and Jiaoyan, et al.
It is worth noting that NeurOWL can be applied iteratively, using predicted missing axioms as new inputs (e.g., PetCat ⊑ Cat or PetCat ⊑ Mammal in the above example), to try to obtain increasingly fine-grained, higher-quality axioms. We discuss case studies to illustrate this potential in Section 6.4.
5
Dataset Construction
We construct a dataset from real-world ontologies to evaluate our method under two settings. datasets to evaluate our method under two settings. Standard Setting In this case, we use all atomic concepts as the candidates set of bridge concept C (i.e, A ⊑ C ⊑ B for given subsumption A ⊑ B) of NeurOWL. The dataset is constructed as follows. First, to simulate ontology incompleteness in a controlled manner, we construct a pruned ontology O′ from the original ontology O by uniformly sampling and removing r = 5% of the TBox axioms. Then, the evaluation dataset with positive and negative samples of subsumptions are constructed as follows: (1) Positive samples. A positive sample is a subsumption A ⊑ B such that O |= A ⊑ B but O′ ̸|= A ⊑ B. We construct the positive sample set P by randomly sampling from all positive candidates. That is, P ⊆ Pall := {(A, B) | O |= A ⊑ B ∧ O′ ̸|= A ⊑ B}. These positive samples may involve one or several bridge concepts, corresponding to downward, upward, or bidirectional cases associated with Stages 2a, 2b, and 3a, respectively. In such cases, the induced missing axioms, together with associated justifications for Stages 2a and 2b, serve as the ground-truth explanations. Alternatively, they may involve no bridge and are therefore expected to be resolved in Stage 3b; in this case, the subsumptions themselves serve as the ground truth explanation. (2) Negative samples. Negative samples are subsumptions A ⊑ B such that O ̸|= A ⊑ B. To construct a balanced dataset, we sample an equal number of negative instances (i.e, |N | = |P|) by replacing B with a B ′ satisfying O ̸|= A ⊑ B ′ , using the following two strategies: – Random: randomly choose one B ′ from all possible candidates. – Hard: restrict the candidates B ′ to those that are closer to A, in the sense that there exists a path of length at most 5 between A and B ′ in the taxonomy (ignoring the edge direction)3 . If none exists, then choose B ′ randomly. Complex Setting. Under standard settings, NeurOWL always returns subsumptions between atomic concepts as the missing axioms. However, in some cases, the missing axiom may involve a complex concept. For example, an ontology {A ⊑ A1 , B1 ⊑ B2 , ∃r.B2 ⊑ B} may miss plausible axiom A ⊑ ∃r.B1 which allows A ⊑ B to be derived. To evaluate the performance of NeurOWL in such complex cases, we construct incomplete ontologies O∃′ in which a bridge concept are of the form ∃r.B1 for 3
The taxonomy of O is a directed acyclic graph over its concepts, with an edge A → B iff O |= A ⊑ B and no B ′ ̸= A, B satisfies O |= A ⊑ B ′ ⊑ B.
Title Suppressed Due to Excessive Length
9
Table 1: Dataset statistics (1,000 positive and 1,000 negative samples per dataset). #|Pall | denotes the total number of possible positive samples. The “Splits” column reports the number of positive samples that can be solved in Stages 2a, 2b, and 3a, respectively; overlaps between stages may occur. Dataset
Concepts
#|Remove Axioms|
#|Pall |
Splits (2a/2b/3a/3b)
FoodOnA SnomedA Snomed∃
30,995 364,352 364,352
2,244 (5%) 18,300 (5%) 2,000 (∃)
102,872 1,181,331 1,774
813/777/977/23 651/569/897/103 -/-/1000/0
some true non-entailed subsumptions A ⊑ B. That is, O∃′ ̸|= A ⊑ B, while in the original ontology O we have O |= A ⊑ ∃r.B1 ⊑ B for some B1 , and r. Random removal is not suitable for constructing such incomplete ontologies, as it rarely yields the desired non-entailed subsumptions. Instead, we adopt a targeted pruning strategy. Specifically, we rank axioms containing existential restrictions (i.e., axioms in which ∃r.B1 appears) by the number of direct subsumptions lost upon their removal, and select the most impactful ones to construct the pruned ontology O∃′ . Positive and negative samples are then generated in the same manner as before. Note that, in this setting, we focus on evaluating the prediction of complex bridge concepts of the form ∃r.B1 . Therefore, NeurOWL is implemented using only Stages 1 and 3.
6
Evaluation
6.1
Experimental Setting
Datasets We use two real-world ontologies for our evaluation: FoodOn (202208-12 version) [12] and Snomed CT (2023-12-24 version) [7]. FoodOn captures general knowledge in the domain of food and agriculture, whereas Snomed CT provides detailed and specialized terms in the healthcare domain. The datasets are obtained by implementing the data construction procedure described in Section 5 on both ontologies under two settings: (1) In the standard setting where atomic concepts serve as bridge concept candidates, we obtain two datasets, FoodOnA and SnomedA . (2) For the complex case where we consider relation restriction concepts of the form ∃r. B as candidates. This setting is applied only to Snomed CT, producing the dataset Snomed∃ . We do not apply this procedure to FoodOn as it lacks subsumptions in which existential restrictions serve as bridges. The statistics of the generated datasets is shown in Table 1.
10
Hui and Jiaoyan, et al.
Implementation We tested two versions of NeurOWL with OnT4 or pre-trained SBERT5 as the embedding model, both using the all-MiniLM-L12-v2 variant. While SBERT is used directly, OnT is fine-tuned6 on the provided incomplete ontology O′ for 20 epochs, with a learning rate of 10−5 , a training batch size of 256, a clustering loss margin of 3.0, and a centripetal loss margin of 0.5. We adopt Qwen3.5-9B7 as the LLM in our experiments, because it is open-source, which makes work easier to reproduce, and achieves strong performance despite its relatively small size. Unless stated otherwise, we use non-thinking mode, as it delivers both high performance and fast inference speed. Without affecting correctness, for a given subsumption A ⊑ B, we restrict the candidate set to direct children of B in Stage 2a 8 and direct parents of A in Stage 2b. This restriction reduces the search space and improves prediction accuracy. Results using all parents/children of A/B are reported in Appendix A.1. Baselines To the best of our knowledge, no existing methods directly fit our setting. We therefore introduce three simple baselines that assess only the plausibility of given subsumptions and do not generate explanations. The baselines are: (1) LLM-only, which uses the same prompt (see Figure 7 in the Appendix) as in Stage 3b to evaluate the plausibility of subsumptions; and (2) two embeddingbased methods, SBERT and OnT, which classify subsumptions as true or false by assessing whether their similarity (for SBERT) or subsumption (for OnT) score exceeds a threshold. The threshold is selected to maximize the F1-score (defined below) on a validation set randomly sampled from the test set, containing 50 positive and 50 negative samples. For fair comparison, we use the same LLM and the same SBERT and OnT as in NeurOWL. As discussed in Sections 1 and 2, existing abductive reasoning approaches are not included in our comparisons, as they differ fundamentally from our setting: they focus on the logical procedure of extracting possible missing axioms from a predefined hypothesis space and treat the given subsumption as an actual observation. Evaluation Metrics We use the following three metrics: – F1: The standard F1-score that measures the correctness of the True/False prediction, independent of explanations. – X-F1: Requires both correct prediction and correct explanation. Specifically, for each positive sample: (1) if a ground-truth bridge concept exists (Stage 2a/2b/3a), the prediction is correct only if the model outputs True and return one of the ground-truth concepts; (2) otherwise, the prediction 4
https://github.com/HuiYang1997/OnT https://sbert.net/ 6 using package at https://pypi.org/project/ontology-transformer/ 7 https://huggingface.co/Qwen/Qwen3.5-9B 8 A is a direct children of B in O′ if O |= A ⊑ B and there is no A′ such that O |= A ⊑ A′ ⊑ B. Direct parent is defined in the same way. 5
Title Suppressed Due to Excessive Length
11
Table 2: Overall results on all three datasets. Dataset Method
Fine-tuned
Random Neg F1
OnT SBERT FoodOnA LLM-only NeurOWL (OnT) NeurOWL (SBERT) OnT SBERT SnomedA LLM-only NeurOWL (OnT) NeurOWL (SBERT) OnT SBERT Snomed∃ LLM-only NeurOWL (OnT) NeurOWL (SBERT)
Hard Neg
X-F1 X-F1* F1
X-F1 X-F1*
✓
0.910 0.712 0.836
✓
0.960 0.893 0.793 0.845 0.781 0.684 0.960 0.885 0.788 0.844 0.772 0.680
✓
0.943 0.927 0.772
✓
0.968 0.847 0.659 0.862 0.745 0.569 0.970 0.846 0.646 0.856 0.736 0.550
✓
0.897 0.927 0.881
✓
0.924 0.497 0.193 0.872 0.460 0.176 0.906 0.002 0.000 0.862 0.002 0.000
— — —
— — —
— — —
— — —
— — —
— — —
0.663 0.514 0.789
0.692 0.565 0.734
0.612 0.623 0.854
— — —
— — —
— — —
— — —
— — —
— — —
is correct only if the model outputs True at Stage 3b. For each negative sample, the prediction is correct if the model outputs False. – X-F1*: the same as X-F1 but requires all the output bridge concepts are among the ground-truth concepts in (1) in testing the positive sample. Baseline methods are evaluated only using F1, as they cannot predict missing axioms or provide explanations. 6.2
Main Results
Summary The overall results are presented in Table 2. We observe that both variants of NeurOWL consistently outperform all baselines in terms of the standard F1 score. Specifically, in the hard negative setting on SnomedA , NeurOWL (OnT) achieves an F1 score of 0.862, compared to 0.734 for the strongest baseline. This improvement can be attributed to the structure of NeurOWL, which enables it to effectively leverage information from the existing ontology (i.e., Stages 2 and 3), and thus leads to better performance in determining the plausibility of given subsumptions. There is only one exception, which occurs in the random negative setting on Snomed∃ , where SBERT slightly outperforms both variants of NeurOWL in terms of F1. This may be due to the candidates of the relational restriction concept ∃r.B1 being extremely large (i.e., beyond 6,000,000 combinations with 20 roles and over 300,000 concepts), which leads to poor retrieval performance of the possible bridged concepts. This is also reflected in
12
Hui and Jiaoyan, et al.
Table 3: Part 1 candidate ranking detailed breakdown (random negatives, positive samples only, %). Dataset Variant
Stage
H@1 H@5 H@10 H@100 MRR
Stage-2a Stage-2b Stage-3a(only)
0.546 0.964 0.920 1.000 0.429 0.714
0.996 1.000 0.714
1.000 1.000 0.857
0.744 0.960 0.538
Stage-2a NeurOWL (SBERT) Stage-2b Stage-3a(only)
0.483 0.892 0.932 1.000 0.286 0.857
0.940 1.000 0.857
1.000 1.000 0.857
0.676 0.965 0.476
Stage-2a Stage-2b Stage-3a(only)
0.664 0.949 0.886 1.000 0.182 0.409
0.974 1.000 0.591
1.000 1.000 0.727
0.786 0.937 0.292
Stage-2a NeurOWL (SBERT) Stage-2b Stage-3a(only)
0.528 0.886 0.903 1.000 0.318 0.591
0.951 1.000 0.682
0.997 1.000 0.955
0.679 0.948 0.462
NeurOWL (OnT) Stage-3a NeurOWL (SBERT) Stage-3a
0.001 0.052 0.000 0.000
0.122 0.000
0.577 0.001
0.042 0.000
NeurOWL (OnT) FoodOnA
NeurOWL (OnT) SnomedA
Snomed∃
the relatively low X-F1 and X-F1* scores (<0.5) and the retrieval performance reported in Table 5 in the next section. However, NeurOWL still shows superior performance in the more challenging hard negative setting on the same dataset, Snomed∃ , which further demonstrates the effectiveness of our method. We also find that fine-tuning on the target ontology leads to consistent improvements: NeurOWL (OnT) always outperforms NeurOWL (SBERT). The performance gap is relatively small on simpler datasets such as FoodOnA and SnomedA , where only atomic concepts are considered as candidates. In contrast, for more complex datasets like Snomed∃ , which involve existential concepts of the form ∃r.B, NeurOWL (OnT) shows significantly better performance due to its adaptation to the underlying logical structure through fine-tuning. This suggests that, in relatively simple default settings, it is sufficient to implement NeurOWL in a trainingfree manner using the SBERT model. However, when considering more complex cases involving complex concepts, it is essential to use fine-tuned models such as OnT. 6.3
Detailed Analysis
Retrieval performance The retrieval results are summarised in Table 3. We categorise the positive samples into different groups based on whether they have downward/upward bridge concepts in Stage 2a/2b (which may overlap) or only bidirectional bridge concepts in Stage 3a (distinct from the former cases). We observe that the retrieval performance of Stage 2a/2b is substantially higher than that of Stage 3a alone. This is reasonable, as the number of candidates in
Title Suppressed Due to Excessive Length Pipeline Stage Stage 2b Stage 3a
Stage 2a
TP 1,000
954 835
Count
819 679
800
800
800
600
600
400
400
200
200
0
0
311
309 163
153 34
X-F1 X-F1*
INOR (OnT)
F1
X-F1 X-F1*
INOR (SBERT)
F1
X-F1 X-F1*
Count
948 738
494
481
F1
X-F1 X-F1*
INOR (OnT)
X-F1 X-F1*
49
F1
X-F1 X-F1*
F1
600
474
453
400 217
209 7
5 F1
X-F1 X-F1*
INOR (OnT)
F1
X-F1 X-F1*
INOR (SBERT)
58
58
F1
X-F1 X-F1*
INOR (OnT)
52
52
F1
X-F1 X-F1*
Count
600
555
600 337 109 1
INOR (OnT)
F1
F1
Setting X-F1
X-F1 X-F1*
INOR (OnT)
F1
X-F1 X-F1*
INOR (SBERT)
X-F1*
FN
783
800
X-F1 X-F1*
0
TN
1,000
1,000
800
800
600
600
982 982 982
982 982 982
860 861
843
800
F1
52
INOR (SBERT)
F1
FP 1,000
993 993 993
200 58
(b) SnomedA
0
995 995 995
400
200 0
X-F1 X-F1*
TN
600
600
F1
INOR (SBERT)
X-F1*
FN
Pipeline Stage Stage 3a Stage 3b
200
Setting X-F1
X-F1 X-F1*
INOR (OnT)
800
0
F1
INOR (SBERT)
(a) FoodOn Stage 3b A
TP
400
0
800
INOR (SBERT)
874
49
800
200
F1
49
1,000
200
INOR (OnT)
INOR (SBERT)
46
1,000
400
X-F1 X-F1*
X-F1 X-F1*
46
1,000
400
1,000
969 969 969
200 46
FP
738
F1
F1
0
Pipeline Stage Stage 2b Stage 3a
TP 942
200
31
INOR (OnT)
Stage 2a
0
966 966 966
671
400
600
TN 1,000
600
800
X-F1*
FN 1,000
400
1,000
Setting X-F1
1,000
600
F1
F1
FP 951
800
Stage 3b
13
X-F1 X-F1*
INOR (SBERT)
400
400
200
200
0
18 F1
18 X-F1 X-F1*
INOR (OnT)
F1
X-F1 X-F1*
INOR (SBERT)
0
400 126 126 126
157 157 157
F1
F1
X-F1 X-F1*
INOR (OnT)
X-F1 X-F1*
200 0
INOR (SBERT)
F1
X-F1 X-F1*
INOR (OnT)
F1
X-F1 X-F1*
INOR (SBERT)
(c) Snomed∃
Fig. 2: Detailed results across different stages.
Stage 2a/2b (i.e., direct children/parents of B/A) is significantly smaller than the space of all atomic concepts. Furthermore, Stage 2b achieves better performance than Stage 2a due to the smaller number of direct parents compared to direct children (e.g., 1.2 vs. 32.7 on average in SnomedA ). Moreover, we find that, due to the large candidate space of relational restriction concepts ∃r.B1 , the retrieval performance on the Snomed∃ dataset is considerably lower than on the other datasets. By fine-tuning on given incomplete ontologies, OnT achieves much better performance than SBERT; however, there remains a large gap compared to the default case where only atomic concepts are considered as the bridge. Per-stage performance Figure 2 presents a detailed breakdown of the results at each stage, including true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN).
14
Hui and Jiaoyan, et al.
Table 4: Stage contribution ablation. “✓” indicates that the stage is applied, while “–” indicates that it is not. The default setting is shaded. Dataset
FoodOnA
SnomedA
Snomed∃
2a 2b 3a 3b
Random-Neg
Hard-Neg
F1
X-F1 X-F1*
F1
X-F1 X-F1*
✓ − −
− ✓ −
− − ✓
− − −
0.823 0.832 0.779
0.766 0.811 0.650
0.689 0.701 0.367
0.789 0.771 0.723
0.733 0.751 0.600
0.656 0.645 0.334
✓ − −
− ✓ −
− − ✓
✓ ✓ ✓
0.941 0.917 0.900
0.773 0.818 0.656
0.698 0.710 0.378
0.870 0.837 0.821
0.708 0.740 0.588
0.634 0.637 0.332
✓
✓
✓
✓
0.960 0.893
0.793
0.845 0.781
0.684
✓ − −
− ✓ −
− − ✓
− − −
0.781 0.718 0.672
0.689 0.667 0.556
0.478 0.611 0.356
0.743 0.673 0.634
0.653 0.622 0.524
0.448 0.569 0.335
✓ − −
− ✓ −
− − ✓
✓ ✓ ✓
0.922 0.884 0.848
0.742 0.721 0.607
0.548 0.669 0.422
0.856 0.819 0.787
0.681 0.660 0.562
0.497 0.611 0.388
✓
✓
✓
✓
0.968 0.847
0.659
0.862 0.745
0.569
−
−
✓
−
0.463
0.156
0.060
0.447
0.149
0.059
−
−
✓
✓
0.911
0.156
0.060
0.873
0.145
0.057
For both FoodOnA and SnomedA , the majority of TP predictions are generated in Stage 2a, followed by Stages 2b and 3a. In contrast, most FPs originate from Stage 3a. This observation aligns with the retrieval performance reported in Table 3, where Stage 3a exhibits relatively lower performance compared to Stages 2a and 2b. Furthermore, we observe that NeurOWL (SBERT) produces almost no TP predictions under the X-F1 and X-F1* metrics, while NeurOWL (OnT) achieves much better performance, indicating the superiority of fine-tuning in handling complex concepts. Ablation of different stages Table 4 reports the performance obtained when using only subsets of the full pipeline. We find that relying on only one of Stages 2a, 2b, or 3a (i.e., rows marked by a single ✓) generally leads to weaker performance, particularly for Stage 3a, where the X-F1* score on FoodOnA (randomneg) drops sharply from 0.793 to 0.367. For SnomedA and Snomed∃ , combining Stages 2b and 3b yields higher X-F1* scores than using the full pipeline. This may be due to the smaller candidate set obtained by focusing solely on direct parents, which improves the likelihood of achieving full correctness for all predicted bridge concepts. Despite this, the full pipeline still achieves better overall F1 and X-F1 scores, highlighting its stronger overall performance.
Title Suppressed Due to Excessive Length
6.4
15
Case Studies
Single-iteration case We present two illustrative cases: one true positive (TP) and one false positive (FP). Assuming the given subsumption is always A ⊑ B. 1. (TP) In SnomedA , in the case A = Irritant contact blepharoconjunc tivitis and B = Disorder of soft tissue. NeurOWL correctly identifies the subsumption A ⊑ B by discovering the bridge concept C = Dermatitis of eyelid in Stage 2b. The LLM-only baseline incorrectly classifies this subsumption as false, likely due to the large semantic gap between A and B. 2. (FP) In FoodOnA , in the case A = Grammatorcynus and B = swine food product. NeurOWL incorrectly identifies C = vertebrate animal food product as a valid bridge concept in Stage 3a. This is erroneous, as the subsumption C ⊑ B does not hold. Multi-iteration case Here, we consider iteratively applying NeurOWL as mentioned in Section 4, where the output missing subsumption in one iteration is used as the target subsumption in the next iteration. On SnomedA , assume the initial input is A ⊑ B with A = Primary undifferentiated carcinoma of anterior wall of nasopharynx and B = Disorder of nasopharynx, and NeurOWL is performed by four iterations: – Iteration 1: For input A ⊑ B, NeurOWL outputs C1 = Lesion of nasopharynx in Stage 2a, together with the new missing axiom A ⊑ C1 . – Iteration 2: We input the missing axiom A ⊑ C1 obtained in Iteration 1 to NeurOWL again, and it produces C2 = Neoplasm of anterior wall of nasopharynx in Stage 2a, along with the new missing axiom A ⊑ C2 . – Iteration 3: With the input A ⊑ C2 , NeurOWL produces C3 = Malignant tumour of anterior wall of nasopharynx in Stage 2a, together with the new missing axiom A ⊑ C3 . – Iteration 4: With the input A ⊑ C3 , NeurOWL produces C4 = Primary malignant neoplasm of anterior wall of nasopharynx in Stage 2c, along with two new missing axioms A ⊑ C4 and C4 ⊑ C3 . In summary, the four iterations of NeurOWL produces the following subsumption chain: A ⊑ C4 ⊑ C3 ⊑ C2 ⊑ C1 ⊑ B, where all the identified missing axioms are correct and exhibit progressively finer granularity. This example demonstrates the potential of iterative applications of NeurOWL to yield increasingly precise and fine-grained results.
7
Conclusion
In this work, we proposed NeurOWL, an end-to-end framework for explainable neural-symbolic reasoning over incomplete OWL ontologies. By integrating the logical structure and textual information of the ontology via embeddings and
16
Hui and Jiaoyan, et al.
LLM, NeurOWL can infer the plausibility of a concept subsumption axiom and in addition can return a set of axioms as its justification, preferring stated axioms in the ontology when possible with up to two predicted missing axioms. Comprehensive experiments and case studies on three datasets based on real-world ontologies have demonstrated an overall good performance of NeurOWL and the effectiveness of its modules. In future work, we aim to extend NeurOWL to support more complex concepts with logical operators of ∀ and ¬ beyond ∃ considered in this work. We also would like to construct datasets and evaluate its iterative capabilities.
References 1. Alrabbaa, C., Borgwardt, S., Friese, T., Hirsch, A., Knieriemen, N., Koopmann, P., Kovtunova, A., Krüger, A., Popovič, A., Siahaan, I.: Explaining reasoning results for owl ontologies with evee. In: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning. vol. 21, pp. 709–719 (2024) 2. Ashburner, M., Ball, C.A., Blake, J.A., Botstein, D., Butler, H., Cherry, J.M., Davis, A.P., Dolinski, K., Dwight, S.S., Eppig, J.T., et al.: Gene ontology: tool for the unification of biology. Nature genetics 25(1), 25–29 (2000) 3. Baader, F.: The Description Logic handbook: Theory, Implementation and Applications. Cambridge university press (2003) 4. Baader, F., Horrocks, I., Lutz, C., Sattler, U.: An Introduction to Description Logic. Cambridge University Press (2017) 5. Babaei Giglou, H., D’Souza, J., Auer, S.: LLMs4OL: Large language models for ontology learning. In: International semantic web conference. pp. 408–427. Springer (2023) 6. Bienvenu, M.: Complexity of abduction in the el family of lightweight description logics. In: KR. pp. 220–230 (2008) 7. Bos, L., Donnelly, K.: SNOMED-CT: The advanced terminology and coding system for ehealth. Stud Health Technol Inform 121, 279–290 (2006) 8. Chen, J., He, Y., Geng, Y., Jiménez-Ruiz, E., Dong, H., Horrocks, I.: Contextual semantic embeddings for ontology subsumption prediction. World Wide Web 26(5), 2569–2591 (2023) 9. Chen, J., Hu, P., Jiménez-Ruiz, E., Holter, O.M., Antonyrajah, D., Horrocks, I.: Owl2vec*: embedding of OWL ontologies. Mach. Learn. 110(7), 1813–1845 (2021). https://doi.org/10.1007/S10994-021-05997-6, https://doi.org/10. 1007/s10994-021-05997-6 10. Chen, J., Mashkova, O., Zhapa-Camacho, F., Hoehndorf, R., He, Y., Horrocks, I.: Ontology embedding: a survey of methods, applications and resources. arXiv preprint arXiv:2406.10964 (2024) 11. Del-Pinto, W., Schmidt, R.A.: ABox abduction via forgetting in ALC. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp. 2768–2775 (2019) 12. Dooley, D.M., Griffiths, E.J., Gosal, G.S., Buttigieg, P.L., Hoehndorf, R., Lange, M.C., Schriml, L.M., Brinkman, F.S., Hsiao, W.W.: Foodon: a harmonized food ontology to increase global food traceability, quality control and data integration. NPJ Science of Food 2(1), 23 (2018) 13. Du, J., Qi, G., Shen, Y.D., Pan, J.Z.: Towards practical ABox abduction in large description logic ontologies. International Journal on Semantic Web and Information Systems (IJSWIS) 8(2), 1–33 (2012)
Title Suppressed Due to Excessive Length
17
14. Du, J., Wan, H., Ma, H.: Practical TBox abduction based on justification patterns. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 31 (2017) 15. Du, J., Wang, K., Shen, Y.D.: A tractable approach to abox abduction over description logic ontologies. In: Proceedings of the AAAI conference on artificial intelligence. vol. 28 (2014) 16. Elsenbroich, C., Kutz, O., Sattler, U.: A case for abductive reasoning over ontologies. In: Grau, B.C., Hitzler, P., Shankey, C., Wallace, E. (eds.) Proceedings of the OWLED*06 Workshop on OWL: Experiences and Directions, Athens, Georgia, USA, November 10-11, 2006. CEUR Workshop Proceedings, CEUR-WS.org (2006), https://ceur-ws.org/Vol-216/submission_25.pdf 17. Glimm, B., Horrocks, I., Motik, B., Stoilos, G., Wang, Z.: HermiT: an OWL 2 reasoner. Journal of automated reasoning 53(3), 245–269 (2014) 18. Haak, A., Koopmann, P., Mahmood, Y., Turhan, A.Y.: Why not? Developing ABox abduction beyond repairs. arXiv preprint arXiv:2507.21955 (2025) 19. Haifani, F., Koopmann, P., Tourret, S., Weidenbach, C.: Connection-minimal abduction in EL via translation to FOL. In: International Joint Conference on Automated Reasoning. pp. 188–207. Springer (2022) 20. Halland, K., Britz, K.: Abox abduction in ALC using a DL tableau. In: Proceedings of the South African institute for computer scientists and information technologists conference. pp. 51–58 (2012) 21. He, Y., Chen, J., Jiménez-Ruiz, E., Dong, H., Horrocks, I.: Language model analysis for ontology subsumption inference. In: Rogers, A., Boyd-Graber, J.L., Okazaki, N. (eds.) Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023. pp. 3439–3453. Association for Computational Linguistics (2023). https://doi.org/10.18653/V1/2023.FINDINGS-ACL. 213, https://doi.org/10.18653/v1/2023.findings-acl.213 22. He, Y., Yuan, M., Chen, J., Horrocks, I.: Language models as hierarchy encoders. In: Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J.M., Zhang, C. (eds.) Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024 (2024), http://papers.nips.cc/paper_files/paper/2024/hash/ 1a970a3e62ac31c76ec3cea3a9f68fdf-Abstract-Conference.html 23. Jackermeier, M., Chen, J., Horrocks, I.: Dual box embeddings for the description logic EL++ . In: Chua, T., Ngo, C., Kumar, R., Lauw, H.W., Lee, R.K. (eds.) Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024. pp. 2250–2258. ACM (2024). https://doi.org/10.1145/3589334. 3645648, https://doi.org/10.1145/3589334.3645648 24. Kazakov, Y., Krötzsch, M., Simancik, F.: ELK reasoner: architecture and evaluation. In: ORE (2012) 25. Kazakov, Y., Skočovskỳ, P.: Enumerating justifications using resolution. In: International Joint Conference on Automated Reasoning. pp. 609–626. Springer (2018) 26. Klarman, S., Endriss, U., Schlobach, S.: Abox abduction in the description logic. Journal of Automated Reasoning 46(1), 43–80 (2011) 27. Koopmann, P.: Signature-based abduction with fresh individuals and complex concepts for description logics. In: Description Logics (2021) 28. Koopmann, P., Del-Pinto, W., Tourret, S., Schmidt, R.A.: Signature-based abduction for expressive description logics. In: Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning. vol. 17, pp. 592–602 (2020)
18
Hui and Jiaoyan, et al.
29. Kulmanov, M., Liu-Wei, W., Yan, Y., Hoehndorf, R.: EL embeddings: Geometric construction of models for the description logic EL++. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. pp. 6103–6109. International Joint Conferences on Artificial Intelligence Organization. https://doi.org/10.24963/ijcai.2019/845, https://www.ijcai.org/ proceedings/2019/845 30. Kulmanov, M., Smaili, F.Z., Gao, X., Hoehndorf, R.: Semantic similarity and machine learning with ontologies. Briefings in bioinformatics 22(4), bbaa199 (2021) 31. Lo, A., Jiang, A.Q., Li, W., Jamnik, M.: End-to-end ontology learning with large language models. Advances in Neural Information Processing Systems 37, 87184– 87225 (2024) 32. Peñaloza, R.: Axiom pinpointing 49, 162–177 (2020). https://doi.org/10.3233/ SSW200042, https://doi.org/10.3233/SSW200042 33. Pukancová, J., Homola, M.: The aaa abox abduction solver: System description. KI-Künstliche Intelligenz 34(4), 517–522 (2020) 34. Shi, J., Dong, H., Chen, J., Wu, Z., Horrocks, I.: Taxonomy completion via implicit concept insertion. In: Proceedings of the ACM Web Conference 2024. pp. 2159– 2169 (2024) 35. Smaili, F.Z., Gao, X., Hoehndorf, R.: Opa2vec: combining formal and informal content of biomedical ontologies to improve similarity-based prediction. Bioinform. 35(12), 2133–2140 (2019). https://doi.org/10.1093/BIOINFORMATICS/BTY933, https://doi.org/10.1093/bioinformatics/bty933 36. Sun, Y., Hao, X., Sun, K., Xu, Y., Yang, X., Dong, X.L., Tang, N., Chen, L.: Are large language models a good replacement of taxonomies? Proc. VLDB Endow. 17(11), 2919–2932 (2024). https://doi.org/10.14778/3681954.3681973, https: //www.vldb.org/pvldb/vol17/p2919-sun.pdf 37. Wei-Kleiner, F., Dragisic, Z., Lambrix, P.: Abduction framework for repairing incomplete el ontologies: Complexity results and algorithms. In: Proceedings of the AAAI conference on artificial intelligence. vol. 28 (2014) 38. Xiong, B., Potyka, N., Tran, T.K., Nayyeri, M., Staab, S.: Faithiful embeddings for EL++ knowledge bases, https://arxiv.org/abs/2201.09919v2 39. Yang, H., Chen, J., He, Y., Gao, Y., Horrocks, I.: Language models as ontology encoders. In: International Semantic Web Conference. pp. 443–461. Springer (2025) 40. Yang, H., Chen, J., Sattler, U.: Transbox: EL++ -closed ontology embedding. In: THE WEB CONFERENCE 2025 41. Yang, H., Chen, J., Sattler, U.: Large language model for OWL proofs. In: Proceedings of the ACM Web Conference 2026. p. 3952–3963. WWW ’26, Association for Computing Machinery, New York, NY, USA (2026). https://doi.org/10.1145/ 3774904.3792395, https://doi.org/10.1145/3774904.3792395 42. Yang, H., Ma, Y., Bidoit, N.: Hypergraph-based inference rules for computing el+-ontology justifications. In: International Joint Conference on Automated Reasoning. pp. 310–328. Springer (2022) 43. Zhang, X., Lai, H., Meng, Q., Bos, J.: Ontourl: A benchmark for evaluating large language models on symbolic ontological understanding, reasoning and learning. arXiv preprint arXiv:2505.11031 (2025) 44. Zhao, H., Tang, W., Guthrie, D., Sevegnani, M., Flynn, D., Harvey, P.: From subsumption to satisfiability: Llm-assisted active learning for owl ontologies. arXiv preprint arXiv:2604.16672 (2026)
103
# candidates (log scale)
Foodon (A) All descendants Direct children (k=1)
104
102 101 100
0
200
400 600 Sample index Snomed (A)
800
1000
104 103 102 101 100
0
200
All descendants Direct children (k=1) 400 600 800 1000 Sample index
# candidates (log scale)
# candidates (log scale)
# candidates (log scale)
Direct (k = 1) vs. All Descendants/Ancestors in 0 Title Due (positive samples sorted bySuppressed direct count in ascending order)to Excessive Length
19
Foodon (A) All ancestors Direct parents (k=1) 101
100
0
200
400 600 Sample index Snomed (A)
800
1000
800
1000
All ancestors Direct parents (k=1) 101
100
0
200
400 600 Sample index
Fig. 3: Comparison between the number of direct and the all children and parents. Samples are ordered by increasing number of direct parents/children.
A
Extra Results
A.1
Direct Children/Parents vs. All
In this section, we compare the performance impact of using only direct parents/children as candidates in Stage 2a/2b (denoted as DIRECT, the default setting of NeurOWL) versus using all transitive parent/child nodes (denoted as ALL). We first present statistics for the number of direct and transitive parent/child nodes associated with the evaluated positive samples. As shown in Figure 4, the average number of direct parents is approximately 1–1.2, which is substantially lower than the average number of direct children (8.6–32.7). This observation aligns with the structural properties of ontologies: a concept typically has only a few direct parents but may have many direct children. Furthermore, Figure 3 illustrates that the number of transitive (ALL) parents/children can be significantly larger than the number of direct ones, in some cases increasing by up to two orders of magnitude. This highlights the substantial expansion of the candidate space when transitive relationships are considered. In Table 5, we present the retrieval performance under the DIRECT and ALL settings. As expected, the DIRECT setting consistently outperforms the ALL setting across all cases, demonstrating the advantage of restricting candidates to direct parents/children. In Table 6, we report the overall performance of the DIRECT and ALL settings. While the ALL setting occasionally achieves higher F1 scores in certain cases, the DIRECT setting demonstrates superior overall performance. This advantage is particularly evident in the consistently higher X-F1 scores across all cases.
20
Distribution of Direct (k = 1) Candidate Counts on 0 (positive test samples)
Hui and Jiaoyan, et al. Foodon (A)
Foodon (A) Median = 2 Mean = 8.6
300
Frequency
Frequency
400
200 100 0
0
10
150 100 50 0
0
20
40 60 80 100 120 # | direct children of B in O' |
140
Median = 1 Mean = 1.0
0
1 2 3 4 # | direct parents of A in O' | Snomed (A)
160
5
6
Median = 1 Mean = 1.2
300 Frequency
Frequency
200
20 30 40 50 60 70 80 # | direct children of B in O' | Snomed (A) Median = 8 Mean = 32.7
600 500 400 300 200 100 0
200 100 0
0
1
2 3 4 5 6 7 8 # | direct parents of A in O' |
9 10 11
Fig. 4: Distribution of the number of direct children and parents for positive samples in test sets.
Table 5: Comparison of retrieval performance using using direct or all Parent/Child. All the results is shown in (DIRECT /ALL). Dataset
NeurOWL
Stage
(OnT)
2a 2b
54.6 / 17.1 96.4 / 39.2 99.6 / 51.4 100.0 / 82.9 74.4 / 28.4 92.0 / 87.9 100.0 / 96.3 100.0 / 98.2 100.0 / 100.0 96.0 / 91.3
(SBERT)
2a 2b
48.3 / 8.2 89.2 / 17.8 94.0 / 24.8 100.0 / 61.0 67.6 / 14.1 93.2 / 82.5 100.0 / 92.3 100.0 / 97.4 100.0 / 100.0 96.5 / 87.0
(OnT)
2a 2b
66.4 / 30.6 94.9 / 67.1 97.4 / 79.7 100.0 / 98.3 78.6 / 47.2 88.6 / 54.5 100.0 / 77.5 100.0 / 87.0 100.0 / 100.0 93.7 / 64.8
(SBERT)
2a 2b
52.8 / 24.4 88.6 / 51.2 95.1 / 61.6 99.7 / 87.1 67.9 / 36.9 90.3 / 55.2 100.0 / 83.1 100.0 / 91.2 100.0 / 100.0 94.8 / 67.0
FoodOnA
SnomedA
H@1
H@5
H@10
H@100
MRR
However, for the X-F1* metric, the ALL setting outperforms DIRECT in some scenarios, such as the hard negative setting in Ours (SBERT). This may be because the candidates in the DIRECT setting are highly relevant and difficult to distinguish, which can lead the LLM to mistakenly verify them as true bridge concepts. As a result, the X-F1* score decreases, since this metric requires all identified bridge concepts to be correct; such errors therefore reduce overall performance. A.2
Ablation of LLMs, embedding models, and retrieval size
To better understand the contribution of different parts in NeurOWL, we conduct a systematic ablation study along three dimensions: (1) the base embedding model used for OnT; (2) the LLM employed in the verification stage; and (3) the candidate retrieval size k. For the embedding model, we evaluate two alternatives:
Title Suppressed Due to Excessive Length
21
Table 6: Overall results of DIRECT vs. ALL. All the results is shown in (DIRECT /ALL). Dataset
Random Neg
NeurOWL F1
FoodOnA
SnomedA
X-F1
Hard Neg X-F1*
F1
X-F1
X-F1*
96.0/96.8 89.3/87.6 79.3/75.0 84.5/86.9 78.1/78.0
68.4/65.8
(SBERT) 96.0/95.3 88.5/83.9 78.8/75.8 84.4/85.5 77.2/74.3
68.0/66.8
(OnT)
(OnT)
96.8/95.7 84.7/77.6 65.9/64.1 86.2/87.1 74.5/69.5 56.9/56.9
(SBERT) 97.0/96.2 84.6/80.4 64.6/66.9 85.6/87.0 73.6/71.6
55.0/59.0
Table 7: Ablation study results on FoodOnA . Default setting is shaded. Random Neg
Hard Neg
F1
X-F1
F1
10 0.966 10 0.963 10 0.964
0.907 0.901 0.905
0.856 0.799 0.858 0.799 0.857 0.800
(2) LLM
MiniLM-L12 9B (think) 10 0.907 MiniLM-L12 4B 10 0.953 MiniLM-L12 9B 10 0.964
0.862 0.865 0.905
0.855 0.808 0.825 0.740 0.857 0.800
(3) Top-k
MiniLM-L12 9B MiniLM-L12 9B MiniLM-L12 9B
5 0.966 20 0.964 10 0.964
0.900 0.907 0.905
0.860 0.794 0.855 0.799 0.857 0.800
Variant
Embed
LLM (Qwen3.5)
MiniLM-L6 9B (1) Embed MPNet 9B MiniLM-L12 9B
k
X-F1
– all-MiniLM-L6-v2 (384-dimensional, same as all-MiniLM-L12-v2 used by default) and – all-mpnet-base-v2 (768-dimensional). For the LLM, we compare Qwen3.5-9B in thinking mode and Qwen3.5-4B (nonthinking) against the default Qwen3.5-9B (non-thinking). For retrieval, we vary the number of candidates using k ∈ {5, 20}, compared to the default setting of k = 10. The results are summarized in Table 7. We can see that, overall, variations in the embedding model and the choice of k lead to only minor performance differences. The former suggests that all evaluated embedding models are sufficiently expressive for our datasets, while the latter is consistent with the strong retrieval performance reported in Table 3, indicating that changing k has limited impact on the final results. In contrast, the choice of LLM has a more noticeable effect. Specifically, the thinking mode slightly reduces performance on random negative cases, but yields a slight improvement in X-F1 for hard negative cases.
22
Hui and Jiaoyan, et al. LLM Prompt You are an ontology expert. Your task is to determine subsumption (is-a) relationships between concepts. Answer only YES or NO for each question. Do not explain. We want to determine: Is "{A}" a subclass of the following concepts: - {candidate_1} - {candidate_2} ... For each candidate, answer YES if "{A}" is a subclass of it, otherwise NO. 1. Is "{A}" a subclass of "{candidate_1}"? 2. Is "{A}" a subclass of "{candidate_2}"? ... Answer format (one per line): 1. YES/NO 2. YES/NO
Fig. 5: Prompt template used for Stage 2a. The prompts for Stage 2b/3a follow a similar procedure, but with different candidate sets: candidates are parents of “A” for Stage 2b, or all atomic concepts for Stage 3a. The questions ask whether the candidates are subclasses of “B” for Stage 2a, or both directions for Stage 3a.
B
Prompts
The prompts used for the LLMs in Stages 2a/2b and 3a setting are illustrated in Figures 5 and 6, respectively. Figure 5 focuses on the downward stage (i.e., Stage 2a), while Figure 6 shows the prompt used for handling complex cases, where role restrictions of the form ∃r. B are treated as candidate sets in Stage 3a. The prompt for Stage 3b is illustrated in Figure 7.
Title Suppressed Due to Excessive Length
23
LLM Prompt You are an ontology expert analyzing concept relationships. Query: Is "{A}" a subclass of "{B}"? Below are {n} candidate existential restrictions that might justify this classification. Each is of the form "role some Filler".
For each candidate, answer YES if: 1. "{A}" plausibly has this role-filler property 2. This filler captures the essential characteristic that defines membership in "{B}" {numbered list of candidates} Respond in the format: 1. YES/NO 2. YES/NO ... Only output the numbered YES/NO answers, nothing else.
Fig. 6: Prompt template for Stage 3a with relation restriction concepts ∃r.B as candidates. {A} and {B} are replaced with concept labels; {n} with the number of candidates; the numbered list enumerates each candidate as “r some B”.
LLM Prompt You are an ontology expert in the biomedical and food domains. Determine whether the following subsumption relationship holds. Is "{A}" a subclass of "{B}"? Answer with only YES or NO.
Fig. 7: Prompt template for Stage 3b and LLM-only baseline.