AgentMap: Joint Equivalence and Subsumption Discovery for Ontology Matching Yiping Song1
Jiaoyan Chen1
Renate A. Schmidt1
Hui Yang1
Wen Zhang2
Department of Computer Science, The University of Manchester, UK {jiaoyan.chen, renate.schmidt, hui.yang-2}@manchester.ac.uk, 1
School of Software Technology, Zhejiang University, China [email protected]
arXiv:2607.27130v1 [cs.AI] 29 Jul 2026
Abstract Ontology matching (OM) has traditionally been formulated as either equivalence discovery or subsumption matching. The existing OM systems identify only one type of semantic correspondence and cannot simultaneously discover equivalence and subsumption mappings. In this paper, we introduce Hybrid Ontology Matching (HOM), a new OM task that unifies equivalence and subsumption discovery, and accordingly propose a Large Language Model (LLM)-based multi-agent OM framework AgentMap that is implemented by a series of interdependent semantic decisions. Given a concept in the source ontology, AgentMap integrates semantic retrieval, hierarchical search, and collaborative multi-agent LLM reasoning to progressively explore the target ontology, identifying either the equivalent concept, if one exists, or the most fine-grained subsumer. We further extend four OM datasets for a HOM benchmark and evaluate AgentMap under hybrid, equivalence-only, and subsumption-only settings. Experimental results show that AgentMap achieves promising performance on the hybrid setting, and at the same time outperforms equivalence matching and subsumption matching baselines on the equivalence-only and subsumption-only settings, respectively.
Keywords: Ontology Matching, Large Language Models, Subsumption Matching, Multi-Agent Systems
1
Introduction
Ontologies have been widely used for knowledge representation in many domains, such as SNOMED CT for healthcare and FoodOn for food and agriculture [2, 4, 23]. For knowledge integration, reuse and interpretation, ontology matching (OM), which is to discover pairs of concepts across ontologies with a specific relationship like equivalence and subsumption, has been widely investigated [20]. However, different ontologies may use different terminologies and adopt conceptualisations with different levels of granularity, and thus accurate OM is challenging, requiring semantic interpretation beyond lexical and structural matching. OM systems have evolved from traditional lexical matching and rule-based approaches, such as LogMap [14] and AgreementMakerLight (AML) [6], to encoder-based pre-trained language model (PLM)-based approaches, such as BERTMap [8], and more recently to generative LLM-based frameworks, such as GenOM [22]. Most of the current OM systems focus on equivalence matching, although there are also some OM systems like BERTSub [3] that aim to discover concept pairs with 1
subsumption relationships. More importantly, to the best of our knowledge, there is a shortage of OM systems that simultaneously discover equivalence and subsumption mappings. To close this gap, we introduce Hybrid Ontology Matching (HOM), a new OM task that requires jointly discovering equivalence and subsumption correspondences for each source concept (formalised in the Problem Statement). Accordingly, we propose a multi-agent OM framework named AgentMap, which decomposes HOM into a series of interdependent semantic reasoning and decision-making steps. This design is inspired by LLM agent paradigms that interleave reasoning with action [27], refine an initial decision using feedback from subsequent verification steps [17], and route control between specialised agents via handoffs [19]. AgentMap combines embedding-based candidate retrieval, hierarchy-aware LLM reasoning, and lexical matching with conflict resolution, decomposing reasoning into specialised agents for equivalence screening, equivalence verification, and subsumption discovery. We further extend four existing OM datasets involving ontologies of the food and biomedicine domains for a benchmark of the new HOM task. The main contributions of this work can be summarised as follows: • We introduce the new task of Hybrid Ontology Matching (HOM), which requires simultaneously considering equivalence and subsumption mappings. It better reflects the requirements of real-world knowledge integration and reuse than conventional OM tasks. • We develop the framework AgentMap that decomposes the complex HOM task into a set of sub-tasks orchestrated by an agentic workflow, effectively integrating established ontology matching techniques with LLM-based reasoning. • We evaluate AgentMap on four HOM datasets, demonstrating consistent improvements over alternative LLM-based strategies. We further compare AgentMap with both classic and recent OM systems under the conventional equivalence-only and subsumption-only settings, where it also achieves superior performance.
2
Problem Statement
Existing OM research is aimed at either equivalence matching or subsumption matching. In this work, we formulate a new task termed Hybrid Ontology Matching (HOM), where the OM system is expected to identify both equivalence and subsumption mappings simultaneously. In particular, given a source named concept cs from a source ontology Os and a target ontology Ot , HOM aims to identify a mapping represented as a triple m = (cs , ct , r), where ct is a named concept from Ot and r ∈ {equivalence, subsumption} is the semantic relation between cs and ct . The equivalence mapping is prioritised, as it represents more fine-grained semantic association. Namely, the system is expected to identify ct as the equivalent concept of cs in Ot if it exists; otherwise, the system is expected to identify ct as the most specific subsumer of cs in Ot (i.e., cs ⊑ ct and there exists no named concept c′t ̸= ct in Ot such that cs ⊑ c′t ⊑ ct ).
3
AgentMap
AgentMap addresses HOM by restricting LLM reasoning to a progressively refined candidate space, combining embedding-based retrieval with iterative exploration of the ontology hierarchy for efficient mapping discovery. Figure 1 presents the overall architecture of AgentMap, which consists of three modules: 2
AgentMap Framework
Source Concept: 𝒄𝒔 + Target Ontology: 𝑶𝒕
Input
Candidate Update
Output
LLM Reasoning
Final Results (𝒄𝒔 , 𝒄𝒕 , r )
Data Preprocessing and Candidate Retrieval Data Extract Textural Data:
cc
Top-k Candidate Selection:
Cosine Similarity Search
c
Label: Parietaria judaica… Label: Parietaria Synonyms: Wall Label: Parietaria judaica… pellitory (Parietaria judaica… Synonyms: Wall Judaica)… Synonyms: Wall pellitory (Parietaria Judaica)…pellitory (Parietaria Judaica)…
Embedding Model
𝜃
Semantic Embeddings:
Source concept 𝒄𝒔 + Candidate set 𝑪+
[0.12, 0.56, 0.91, 0.21, ...] [0.43, 0.22, 0.78, 0.34, ...] [0.88, 0.34, 0.11, 0.72, ...]
LLM-Based Conflict resolution
Lexical Matcher (Eqv Only)
Agent-Based Result
Agent : Equivalence Verification
Agent : Initial Equivalence Screening
Identify Potential Equivalent Concept
Lexical -Based Result
Agent-Based Reasoning
Source concept 𝒄𝒔 + Candidate set 𝑪𝟎
LLM Reasoning
Lexical Matching and Conflict Resolution
Candidate set 𝑪+ Candidate set 𝑪𝟎
Yes Is equivalence found?
Source concept 𝒄𝒔 + Candidate concept 𝒄ො 𝒕
No Source concept 𝒄𝒔 + Candidate set 𝑪𝟎
Candidate Update
Source concept 𝒄𝒔 + Candidate set 𝑪𝒓𝒆𝒇
Retrieve Parents and Children
LLM Reasoning Identify Closest Valid Subsumer
LLM Reasoning Verify Equivalence Using Ontology Hierarchy
Agent : Subsumption Discovery
No
Source concept 𝒄𝒔 + Candidate set 𝑪𝒊+𝟏
Output Best Subsumer
Yes
Is subsumer found? Expand candidates 𝑪𝒊+𝟏 = Parents(𝑪𝒊 )
LLM Reasoning If 𝒊 ≥ 𝒅𝒎𝒂𝒙 , Max iteration reached.
Identify the Closest Subsumer from Visited Concepts
Figure 1: Overview of the proposed AgentMap framework. Semantic retrieval first constructs task-specific candidate sets for the agent-based reasoning and lexical matching modules. Their predictions are subsequently reconciled through LLM-based conflict resolution to jointly predict the target concept and semantic relation. Data Preprocessing and Candidate Retrieval extracts textual information from the source concept and the target ontology, encodes concepts into semantic embeddings, performs cosinesimilarity-based retrieval, and constructs two candidate sets for lexical matching and agent-based reasoning, respectively. Agent-Based Reasoning performs collaborative reasoning through multiple LLM-based agents and ontology-aware search to progressively identify semantic correspondences between concepts. Lexical Matching and Conflict Resolution complements agent-based semantic reasoning with lexical evidence and produces a unified correspondence by resolving potential inconsistencies between the two matching strategies.
3.1
Data Preprocessing and Candidate Retrieval
Given a source concept cs and a target ontology Ot , this module constructs two task-specific candidate sets for the agent and lexical matching modules. AgentMap extracts the textual information of cs and the concepts in Ot , specifically the label and synonyms of each concept, encodes them into semantic embeddings, and ranks the target concepts according to their cosine similarity to cs . Two candidate sets are then constructed from this ranking using different top-k thresholds: (1) A smaller candidate set C0 is constructed for the agent module, limiting costly LLM reasoning to the most semantically relevant concepts. (2) A larger candidate set C + is constructed for lexical matching, whose low computational cost permits broader candidate coverage. This dual candidate-set design provides each downstream module with a candidate scope 3
suited to its computational characteristics and matching objective.
3.2
Agent-Based Reasoning
The agent-based reasoning module consists of three specialised agents: AgentES for initial equivalence screening, AgentEV for equivalence verification, and AgentSD for subsumption discovery. The module follows an equivalence-first strategy. AgentES first examines the initial candidate set C0 for a potential equivalent concept. If an equivalence candidate is identified, AgentEV verifies the prediction through an ontology-structure-guided candidate update. Otherwise, AgentSD performs iterative subsumption discovery by progressively expanding the candidate set along the target ontology hierarchy. Equivalence Discovery. AgentES receives the source concept cs and the initial candidate set C0 , and performs LLM reasoning to determine whether C0 contains a valid equivalent target concept. If an equivalence candidate is identified, AgentES outputs the most likely target concept ĉt and passes it to AgentEV for further verification. AgentEV receives the source concept cs together with a candidate set Cref , constructed from the direct parent and child concepts of ĉt in the target ontology. Specifically, AgentMap retrieves the direct parent and child concepts of ĉt , denoted P (ĉt ) and Ch(ĉt ), respectively, and, to avoid introducing every parent and child concept into the verification process, selects only the parent concept and the child concept that are most semantically similar to cs : p∗ = arg max sim(cs , p), p∈P (ĉt )
ch∗ = arg
max ch∈Ch(ĉt )
sim(cs , ch),
where sim(·, ·) denotes cosine similarity of the embeddings. The resulting candidate set is Cref = {ĉt , p∗ , ch∗ }. If ĉt has no direct parent concept (respectively, no direct child concept), p∗ (respectively, ch∗ ) is undefined and is simply omitted from Cref ; in this case Cref contains only ĉt together with whichever of p∗ and ch∗ exists. AgentEV then performs LLM reasoning over Cref to verify the equivalence correspondence, comparing ĉt against its most semantically relevant parent and child concepts. Based on this comparison, AgentEV outputs a single final candidate from Cref : it either retains ĉt , or replaces it with p∗ or ch∗ , whichever candidate’s granularity better matches cs . Subsumption Discovery. If AgentES does not identify a valid equivalent concept, AgentMap invokes AgentSD to iteratively search for the closest valid subsumer. AgentSD initially receives the source concept cs together with the initial candidate set C0 . At ith iteration, it performs LLM reasoning over the current candidate set Ci to determine whether any candidate is a valid subsumer of cs . If a valid subsumer ct is identified, AgentMap returns (cs , ct , subsumption). Otherwise, the candidate set is updated using the structure of the target ontology. Specifically, all direct parents of each concept in Ci are collected to form the candidate set for the next iteration: [ Ci+1 = P arents(Ci ) = P (c). c∈Ci
4
AgentSD then performs the same reasoning process over Ci+1 . Therefore, each iteration examines the direct parents of the complete candidate set from the preceding iteration, allowing the search to move upward through the ontology hierarchy level by level. This ontology-structure-guided update continues until a valid subsumer is identified or the maximum iterations dmax is reached. If no valid subsumer is found after dmax upward expansions, AgentSD performs a final LLM reasoning step over all candidates visited throughout the search process: C0 ∪ C1 ∪ · · · ∪ Cdmax . The agent then selects the most appropriate subsumer from these visited concepts.
3.3
Lexical Matching and Conflict Resolution
In parallel with the agent-based reasoning module, AgentMap performs lexical matching to identify equivalence correspondences supported by lexical evidence. The lexical matching module operates on the candidate set C + generated during candidate retrieval. For each candidate concept in C + , the module compares its representations with the source concept, including labels and synonyms. A candidate is regarded as a lexical match if one of its labels or synonyms matches with some label or synonym of the source concept. Because lexical matching alone cannot determine hierarchical relations, this module predicts only equivalence correspondences. In the end, the lexical matching result and the agent reasoning result are reconciled through LLM-based conflict resolution if they are not identical: (i) if no candidate in C + satisfies the lexical matching criterion above (i.e., the lexical matching module identifies no equivalence candidate), AgentMap returns the agent reasoning result; (ii) if lexical matching outputs an equivalent concept while the agents infer a subsumer, AgentMap returns the lexical matching result, following the equivalence-first problem setting; (iii) if lexical matching and agent reasoning output two different equivalent target concepts, AgentMap performs an additional LLM reasoning step to determine the final output.
4
Evaluation
4.1
Benchmark and Metrics
Benchmark Construction Existing benchmarks, such as Bio-ML used by Ontology Alignment Evaluation Initiative (OAEI) [10], provide equivalence and subsumption reference mappings independently. They can only evaluate OM systems for either equivalence or subsumption matching. We therefore construct a dedicated benchmark for the new task of HOM. The proposed benchmark is built upon four equivalence OM datasets (tasks), including three from OAEI Bio-ML (SNOMED–FMA–Body, SNOMED–NCIT–Pharm, and NCIT–DOID–Disease) which match medical ontologies, and the HeLiS-FoodOn dataset [3] which mathches a health lifestyle ontology with the food ontology. Since HOM requires jointly evaluating equivalence and subsumption matching against a single target ontology, we reorganize the source concepts of the original equivalence benchmark into two disjoint subsets: one for equivalence evaluation and one for subsumption evaluation. Subsumption ground truth, however, is not independently annotated; instead, following [10], it is derived from the equivalence mappings by taking the direct parent of each equivalence target as the corresponding subsumer. This construction creates an ambiguity: for concepts assigned to the subsumption subset, their true equivalence target is still present in the original target ontology, even though the intended ground truth is now its parent. Since the equivalence target is, by definition, a more specific and 5
Dataset
#Teq
#Tsub
#Ttest
SNOMED–FMA–Body SNOMED–NCIT–Pharm NCIT–DOID–Disease HeLiS–FoodOn
3099 1120 2920 174
3072 3545 1694 190
6171 4665 4614 364
Table 1: Statistics of the constructed HOM benchmark. semantically closer match than its parent, any system able to identify it would report it instead of the coarser subsumption target used as ground truth, rendering subsumption evaluation ill-defined on the unmodified ontology. To resolve this, we remove, from the original target ontology, only the equivalence target concepts of the source concepts assigned to the subsumption subset, and reattach their child concepts to the corresponding parent to preserve the original taxonomy. The resulting ontology, denoted Ot , is used consistently by all methods (AgentMap and baselines) throughout the paper, and jointly supports equivalence-only, subsumption-only, and HOM evaluation on a single, shared target ontology. Each resulting HOM dataset consists of a source ontology, a target ontology and a test set of source concepts Ttest , each of which is annotated by exactly one ground-truth target concept and the corresponding matching type (equivalence or subsumption). The testing subset with equivalence (resp. subsumption) target concept is denoted as Teq (resp. Tsub ). See Table 1 for more dataset statistics. Evaluation Metrics Three evaluation settings are considered: HOM, equivalence-alone, and subsumption-alone. The HOM setting uses the whole test set Ttest . A source concept is regarded as correctly processed only if both the target concept and the matching type are correctly identified. The overall performance is measured by OverallAcc. which is the ratio of the corrected predicted source concepts among all the source concepts Ttest . To further analyse performance on different matching types, we additionally report the accuracies EqvAcc. and SubAcc. on the equivalence and subsumption subsets of the whole testing set, i.e., Teq and Tsub , respectively. In the HOM setting, the system does not know the matching type, while in the equivalence or subsumption-alone setting, we let the system know the matching type in advance, so as to fairly comparing AgentMap with existing OM systems for either equivalence or subsumption matching. A source concept is regarded as correctly processed if its ground-truth target concept is identified. The equivalence matching performance is measured by Accuracyeq , which is the ratio of correctly processed source concepts among Teq , and the subsumption matching performance is measured by Accuracysub which is the ratio of correctly processed source concepts among Tsub .
4.2
Baselines
HOM Setting. We construct four LLM-based baselines by varying the prompting strategy and the available candidate set. Two prompting strategies are considered: direct prompting and Chainof-Thought (CoT) prompting [24]. Two candidate configurations are evaluated. The first uses the same reasoning candidate set C0 as AgentMap, allowing the LLM to directly predict both the target concept and the semantic relation. The second expands C0 by including the parent concepts (up to two ontology levels) and direct child concepts of each retrieved candidate, forming an expanded neighbourhood candidate set that approximates the maximum ontology neighbourhood explored by AgentMap. 6
Combining the two prompting strategies with the two candidate configurations yields four baselines: • LLM + C0 : Direct prompting over the reasoning candidate set. • LLM + C0 + CoT: Chain-of-Thought prompting over the reasoning candidate set. • LLM + Neighbour: Direct prompting over the expanded neighbourhood candidate set. • LLM + Neighbour + CoT: Chain-of-Thought prompting over the expanded neighbourhood candidate set. These baselines control both the prompting strategy and the available candidate coverage, helping isolate the contribution of AgentMap’s staged multi-agent reasoning process. Equivalence-alone Setting. For equivalence OM, we compare AgentMap with representative methods from three categories. Traditional OM systems, LogMap [14] and AML [6], take two complete ontologies as input and produce a single global alignment for the whole ontology pair rather than a per-concept prediction. To adapt them to Teq , we look up each source concept in this global alignment and use its mapped target, if any, as the prediction. In contrast, pretrained language model methods, represented by BERTMap [8], and LLM methods, represented by GenOM [22], natively predict a target concept for each source concept individually, and are therefore evaluated directly on Teq following the same protocol as AgentMap. Subsumption-alone Setting. For subsumption OM, we compare AgentMap with five representative methods that score each candidate subsumer in an embedding space and rank them accordingly: (1) General sentence embedding methods for semantic similarity, such as SBERT and OpenAI text-embedding-3-small Embedding; (2) Fine-tuned hirerachy embeddings methods, such as HiT [11] and OnT [26]; and (3) Fine-tuned PLM-based classification model, such as BERTSub [3], which is a representative subsumption OM system. For all baselines, accuracy is computed as Hit@1, i.e., whether the top-scored candidate matches the ground truth. Among these baselines, embedding-based methods encode each concept once and reuse the encoding for every comparison, and can therefore consider all named concepts as candidates; BERTSub instead reruns the model for every source–candidate pair, making exhaustive scoring over the full ontology infeasible, and therefore requires a predefined candidate list as input. To ensure a fair comparison, we provide BERTSub with a candidate set consisting of the initial retrieved candidates C0 together with their parent concepts expanded up to two ontology levels, matching the maximum search range explored by AgentSD during subsumption discovery. This controls candidate coverage while allowing different ranking and reasoning strategies to be compared.
4.3
Experimental Setup
Unless otherwise specified, AgentMap uses GPT-4.1-mini as the backbone LLM throughout all experiments. All LLM-based methods are evaluated with a decoding temperature of 0. Ontology concepts are encoded using the OpenAI text-embedding-3-small embedding model, and cosine similarity is used for candidate retrieval. The agent-based reasoning module operates on the top-5 retrieved candidates (C0 ), while the lexical matching module uses the top-20 retrieved candidates (C + ). During ontology-guided subsumption search, the maximum upward traversal dmax is set to 2 (i.e., up to the grandparent level of the initially retrieved target candidates in the ontology 7
Task
System
OverallAcc. EqvAcc. SubAcc.
LLM+C0 SNOMED LLM+C0 +CoT -FMA LLM+Neighbour -Body LLM+Neighbour+CoT
0.536 0.546 0.581 0.634
0.936 0.940 0.944 0.949
0.132 0.149 0.215 0.318
AgentMap
0.657
0.957
0.354
LLM+C0 SNOMED LLM+C0 +CoT -NCIT LLM+Neighbour -Pharm LLM+Neighbour+CoT
0.255 0.258 0.374 0.454
0.981 0.976 0.978 0.976
0.026 0.031 0.184 0.289
AgentMap
0.523
0.981
0.378
LLM+C0 LLM+C0 +CoT LLM+Neighbour LLM+Neighbour+CoT
0.728 0.736 0.760 0.772
0.940 0.930 0.952 0.944
0.362 0.400 0.430 0.476
AgentMap
0.787
0.944
0.513
LLM+C0 LLM+C0 +CoT LLM+Neighbour LLM+Neighbour+CoT
0.387 0.418 0.434 0.461
0.713 0.718 0.793 0.793
0.089 0.142 0.105 0.158
AgentMap
0.452
0.753
0.174
NCIT -DOID -Disease
HeLiS -FoodOn
Table 2: HOM Results. All the methods (AgentMap and the baselines) use GPT-4.1-mini as the backbone LLM. Bold indicates the best result and underline indicates the second-best in each column. hierarchy). The same configuration is adopted throughout all experiments unless explicitly stated otherwise. To evaluate the robustness of AgentMap, we additionally replace GPT-4.1-mini with several representative open-source LLMs. 1
4.4
Experimental Results
HOM Results. Table 2 reports results for the HOM setting. AgentMap achieves the highest overall accuracy on the three biomedical benchmarks (SNOMED-FMA-Body, SNOMED-NCIT-Pharm, and NCIT-DOID-Disease), and is only slightly behind the best baseline method on HeLiS–FoodOn. Specifically, AgentMap achieved consistently best subsumption accuracy across all four datasets. For example, on SNOMED-NCIT-Pharm, AgentMap’s subsumption accuracy is up to 30.8% higher than the best baseline (0.378 vs. 0.289). These results show that decomposing HOM into staged semantic decisions, combined with iterative ontology-guided search, is substantially more effective than single-step reasoning over a fixed candidate set. For equivalence accuracy, the gap between AgentMap and the other baselines is much smaller. In some cases, such as HeLiS–FoodOn, LLM+Neighbourhood achieves higher equivalence accuracy than AgentMap. This is due to the fact that the baseline’s expanded neighbourhood decides equiv1 See Appendix for LLM prompt templates, SBERT embedding results, C0 size sensitivity, and further details; code and data will be released.
8
SNOMED-FMA
SNOMED-NCIT
Method
Acceq
Method
Acceq
LogMap AML BERTMap GenOM(Qwen32B)
0.470 0.906 0.598 0.774
LogMap AML BERTMap GenOM(Qwen32B)
0.901 0.919 0.809 0.888
AgentMap(Qwen32B) 0.949 AgentMap(GPT) 0.957
AgentMap(Qwen32B) 0.978 AgentMap(GPT) 0.981
NCIT-DOID
HeLiS-FoodOn
Method
Acceq
Method
Acceq
LogMap AML BERTMap GenOM(Qwen32B)
0.651 0.780 0.779 0.889
LogMap AML BERTMap GenOM(Qwen32B)
0.626 0.701 0.391 0.707
AgentMap(Qwen32B) 0.942 AgentMap(GPT) 0.944
AgentMap(Qwen32B) 0.753 AgentMap(GPT) 0.753
Table 3: Comparison with existing ontology matching system on equivalence alignment. Bold indicates the best result and underline the second-best in each column. alence and subsumption together in a single LLM call, so the candidate set it sees when making the equivalence decision already includes the parent, grandparent, and child concepts needed for subsumption. AgentMap’s AgentES , in contrast, judges equivalence using only C0 , which is a much smaller set. With only 174 equivalence instances in HeLiS–FoodOn, this gap in candidate coverage is further amplified, noticeably lowering AgentMap’s equivalence accuracy relative to the baseline. Equivalence-alone Results. Table 3 compares AgentMap with representative OM systems on equivalence-alone matching. As GenOM relies on next-token probabilities for candidate selection, it is only evaluated with open-source LLMs; we thus additionally report AgentMap under the same backbone (Qwen2.5-32B-Instruct [25]) for a fair comparison, retaining GPT-4.1-mini as the default elsewhere. AgentMap achieves the best performance on all four benchmarks under both backbones. LogMap fluctuates sharply (0.470 to 0.901): as an earlier-generation system, it relies heavily on external lexicons for matching, whose coverage varies across domains, unlike BERT- and LLM-based methods that draw on learned semantic representations instead. BERTMap is competitive on the three medical benchmarks but collapses to 0.391 on HeLiS–FoodOn: it is built on BioClinicalBERT, a BERT variant pre-trained specifically on clinical and biomedical text, whose vocabulary and representations are tailored to medical terminology and therefore transfer poorly to the food and lifestyle domain of HeLiS–FoodOn. GenOM stays comparatively stable across all four, consistent with LLMs carrying broader, less domain-specific knowledge. Switching AgentMap’s backbone from Qwen2.5-32B to GPT-4.1-mini yields only modest further gains, indicating the improvement stems mainly from the reasoning framework rather than the backbone LLM. Subsumption-alone Results. Table 4 presents results that compare AgentMap with the subsumption OM baselines. AgentMap achieves the highest accuracy on all four benchmarks, with gains most pronounced on the three biomedical benchmarks; it improves the accuracy over the best 9
SNOMED–FMA–Body
SNOMED–NCIT–Pharm
Method
Accsub
Method
Accsub
OnT HiT SBERT OpenAI BERTSub
0.054 0.037 0.063 0.076 0.191
OnT HiT SBERT OpenAI BERTSub
0.012 0.003 0.004 0.009 0.046
AgentMap
0.401
AgentMap
0.398
NCIT–DOID–Disease
HeLiS–FoodOn
Method
Accsub
Method
Accsub
OnT HiT SBERT OpenAI BERTSub
0.225 0.261 0.263 0.246 0.336
OnT HiT SBERT OpenAI BERTSub
0.190 0.100 0.090 0.074 0.269
AgentMap
0.564
AgentMap
0.278
Table 4: Comparison with existing subsumption matching methods. Bold indicates the best result and underline the second-best in each column. baseline BERTSub from 0.191 to 0.401 on SNOMED-FMA-Body, from 0.046 to 0.398 on SNOMEDNCIT-Pharm, and from 0.336 to 0.564 on NCIT-DOID-Disease. There is also a small gain (from 0.269 to 0.278) on HeLiS-FoodOn.Note that BERTSub is given the the initial retrieved candidates C0 as AgentMap (see Baselines Section). The gain is especially large on SNOMED-NCIT-Pharm, where the top-ranked candidates are often near-synonymous, making it hard for embedding- or BERT-based scoring to separate the correct subsumer from its closest competitors; AgentMap’s LLM-based reasoning is better able to resolve such fine-grained distinctions. More broadly, these results show that purely ranking is insufficient for subsumption matching and it requires progressively exploring the hierarchy through iterative agent-based reasoning.
4.5
Ablation Study
Table 5 reports ablation results on SNOMED-FMA-Body, the largest of the four benchmarks, examining two components: the hierarchical search mechanism and the Lexical Matching and Conflict Resolution (LM&CR) module. The hierarchical search appear to be the primary contributor to AgentMap’s subsumption performance. Removing hierarchical search can reduces overall accuracy by 15.7% and subsumption accuracy by more than half (58.2%), while equivalence accuracy is unaffected, Removing LM&CR, in contrast, causes only a modest drop in overall (1.4%) and equivalence accuracy (1.7%), with no effect on subsumption, indicating that LM&CR mainly refines equivalence prediction.
4.6
Effect of Backbone LLMs
Table 6 evaluates the effect of backbone LLMs on AgentMap. The framework performs stably across LLMs such as GPT-4.1-mini, Qwen2.5-32B-Instruct, Qwen2.5-72B-Instruct [25], and Llama3.1-70BInstruct [5], indicating that its effectiveness is largely independent of the backbone. GPT-4.1-mini 10
Method
OverallAcc.
EqvAcc.
SubAcc.
AgentMap 0.657 0.957 0.354 w/o LM&CR 0.648 (-1.4%) 0.941 (-1.7%) 0.354 (-0.0%) w/o Hierarchical Search 0.554 (-15.7%) 0.957 (-0.0%) 0.148 (-58.2%)
Table 5: Ablation study of AgentMap on the SNOMED-FMA-Body benchmark (LM&CR denotes the Lexical Matching and Conflict Resolution module). Task
LLM
SNOMED– FMA-Body
GPT-4.1-mini Qwen2.5-32B-Instruct Qwen2.5-72B-Instruct Llama3.1-70B-Instruct Mixtral-8x7B-Instruct
0.657 0.635 0.639 0.641 0.572
0.957 0.949 0.955 0.960 0.936
0.354 0.318 0.320 0.318 0.205
GPT-4.1-mini Qwen2.5-32B-Instruct SNOMED– NCIT-Pharm Qwen2.5-72B-Instruct Llama3.1-70B-Instruct Mixtral-8x7B-Instruct
0.523 0.528 0.520 0.509 0.459
0.981 0.978 0.979 0.979 0.973
0.378 0.386 0.375 0.361 0.296
GPT-4.1-mini Qwen2.5-32B-Instruct NCIT–DOID Qwen2.5-72B-Instruct -Disease Llama3.1-70B-Instruct Mixtral-8x7B-Instruct
0.787 0.772 0.777 0.755 0.693
0.944 0.942 0.941 0.940 0.914
0.513 0.478 0.493 0.434 0.311
GPT-4.1-mini Qwen2.5-32B-Instruct Qwen2.5-72B-Instruct Llama3.1-70B-Instruct Mixtral-8x7B-Instruct
0.452 0.434 0.453 0.405 0.371
0.753 0.753 0.753 0.730 0.695
0.174 0.142 0.179 0.106 0.074
HeLiS– FoodOn
OverallAcc. EqvAcc. SubAcc.
Table 6: Effect of different backbone LLMs on HOM, Bold indicates the best result. and the Qwen2.5 variants are the strongest overall, each best on two of the four benchmarks, with Llama3.1-70B-Instruct remaining competitive; Superisingly, in SNOMED-NCIT-Pharm task, Qwen2.5-32B even archives better performance than bigger Qwen2.5-72B model. Mixtral-8x7BInstruct [13], in contrast, consistently underperforms across all four, which may be due to its weaker capabilities.
5
Related Work
5.1
Equivalence Ontology Matching
Equivalence OM has evolved from rule-based systems to neural representation learning and, more recently, LLM-based reasoning. Early systems, such as LogMap [14] and AML [6], combine lexical similarity, ontology structures, and logical reasoning to construct high-quality ontology alignments, and remain strong baselines in the OAEI benchmark. With the development of deep learning, OM has increasingly relied on learned semantic representations [8]. Early neural approaches employed convolutional neural networks (CNNs) to encode 11
ontology concepts from textual descriptions [1]. More recent methods adopt transformer-based language models, including BERTMap [8], BioSTransformer [18], BioGITOM [21], and Magneto [15], which leverage contextual language representations, graph neural architectures, or hybrid smalllarge language models to improve semantic matching accuracy. Recent advances in LLMs have further shifted OM from representation learning toward semantic reasoning [9]. Representative systems, including GenOM [22], LogMap-LLM [16], Olala [12], and LLM4OM [7], exploit the reasoning capability of LLMs to perform ontology alignment through semantic understanding and multi-step inference. Although these methods differ substantially in architecture, they are all designed for equivalence OM, where the objective is to identify concepts referring to the same real-world entity across heterogeneous ontologies.
5.2
Subsumption Ontology Matching
Compared with equivalence OM, subsumption OM has received considerably less attention. Existing studies commonly formulate the task as a candidate ranking problem [10], where the objective is to identify the correct subsumer from a predefined candidate list. BERTSub [3] is one of the few approaches specifically designed for subsumption OM. It uses contextual language models to rank candidate subsumers under existing benchmark settings. However, BERTSub assumes that the benchmark-supported subsumer is already contained in the candidate list and therefore evaluates candidate ranking rather than ontology-wide subsumption discovery. Ontology representation learning methods such as OnT [26] and HiT [11] have also been used as concept encoders in related ranking settings. However, they are not designed specifically for subsumption OM. In this work, we include them only as encoder-based baselines to examine how ontology-aware representations perform under our evaluation protocol. HOM instead requires searching the target ontology to jointly identify the target concept and determine whether the relation is equivalence or subsumption — a joint formulation that, to the best of our knowledge, no existing OM framework supports.
6
Conclusion
We introduced Hybrid Ontology Matching (HOM), reframing equivalence and subsumption discovery as a single task evaluated jointly, reflecting the fact that, in practice, whether a source concept has an exact match or only a broader one is not known in advance. AgentMap addresses HOM by decomposing this joint decision into staged, interdependent agent reasoning steps rather than a single LLM judgment. Our ablation and cross-backbone experiments show that this staged decomposition, rather than candidate coverage or backbone strength, drives AgentMap’s gains, suggesting iterative, structure-aware reasoning as a general principle for LLM agents over hierarchical structures. AgentMap also sets a new state of the art on subsumption matching, though its absolute accuracy remains below 0.5 on three of four benchmarks, showing this sub-task is still considerably harder than equivalence matching. Future work includes closing this gap through more targeted hierarchical search and extending HOM to richer semantic relations.
References [1] Alexandre Bento, Amal Zouaq, and Michel Gagnon. Ontology matching using convolutional neural networks. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, 12
Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5648–5653, Marseille, France, May 2020. European Language Resources Association. [2] L Bos and K Donnelly. Snomed-ct: The advanced terminology and coding system for ehealth. Stud Health Technol Inform, 121:279–290, 2006. [3] Jiaoyan Chen, Yuan He, Yuxia Geng, Ernesto Jiménez-Ruiz, Hang Dong, and Ian Horrocks. Contextual semantic embeddings for ontology subsumption prediction. World Wide Web, 26(5):2569–2591, September 2023. [4] Damion M Dooley, Emma J Griffiths, Gurinder S Gosal, Pier L Buttigieg, Robert Hoehndorf, Matthew C Lange, Lynn M Schriml, Fiona SL Brinkman, and William WL Hsiao. Foodon: a harmonized food ontology to increase global food traceability, quality control and data integration. npj Science of Food, 2(1):23, 2018. [5] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models, 2024. [6] Daniel Faria, Catia Pesquita, Emanuel Santos, Matteo Palmonari, Isabel F. Cruz, and Francisco M. Couto. The AgreementMakerLight Ontology Matching System. In Robert Meersman, Hervé Panetto, Tharam Dillon, Johann Eder, Zohra Bellahsene, Norbert Ritter, Pieter De Leenheer, and Deijing Dou, editors, On the Move to Meaningful Internet Systems: OTM 2013 Conferences, pages 527–541, Berlin, Heidelberg, 2013. Springer. [7] Hamed Babaei Giglou, Jennifer D’Souza, Felix Engel, and Sören Auer. LLMs4OM: Matching Ontologies with Large Language Models, April 2024. arXiv:2404.10317. [8] Yuan He, Jiaoyan Chen, Denvar Antonyrajah, and Ian Horrocks. BERTMap: A BERT-Based Ontology Alignment System. Proceedings of the AAAI Conference on Artificial Intelligence, 36(5):5684–5691, June 2022. Number: 5. [9] Yuan He, Jiaoyan Chen, Hang Dong, and Ian Horrocks. Exploring Large Language Models for Ontology Alignment, September 2023. arXiv:2309.07172. [10] Yuan He, Jiaoyan Chen, Hang Dong, Ernesto Jiménez-Ruiz, Ali Hadian, and Ian Horrocks. Machine Learning-Friendly Biomedical Datasets for Equivalence and Subsumption Ontology Matching. In Ulrike Sattler, Aidan Hogan, Maria Keet, Valentina Presutti, João Paulo A. Almeida, Hideaki Takeda, Pierre Monnin, Giuseppe Pirrò, and Claudia d’Amato, editors, The Semantic Web – ISWC 2022, pages 575–591, Cham, 2022. Springer International Publishing. [11] Yuan He, Zhangdie Yuan, Jiaoyan Chen, and Ian Horrocks. Language models as hierarchy encoders. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA, 2024. Curran Associates Inc. [12] Sven Hertling and Heiko Paulheim. OLaLa: Ontology Matching with Large Language Models. In Proceedings of the 12th Knowledge Capture Conference 2023, K-CAP ’23, pages 131–139, New York, NY, USA, December 2023. Association for Computing Machinery.
13
[13] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts, 2024. [14] Ernesto Jiménez-Ruiz and Bernardo Cuenca Grau. Logmap: Logic-based and scalable ontology matching. In The Semantic Web–ISWC 2011, volume 7031 of Lecture Notes in Computer Science, pages 273–288. Springer, 2011. [15] Yurong Liu, Eduardo Pena, Aecio Santos, Eden Wu, and Juliana Freire. Magneto: Combining Small and Large Language Models for Schema Matching, June 2025. arXiv:2412.08194 [cs]. [16] Sviatoslav Lushnei, Dmytro Shumskyi, Severyn Shykula, Ernesto Jiménez-Ruiz, and Artur d’Avila Garcez. Large language models as oracles for ontology alignment. In Vera Demberg, Kentaro Inui, and Lluís Marquez, editors, Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2435–2449, Rabat, Morocco, March 2026. Association for Computational Linguistics. [17] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems, 36:46534–46594, 2023. [18] Safaa Menad, Wissame Laddada, Saïd Abdeddaïm, and Lina Soualmia. BioSTransformers for Biomedical Ontologies Alignment:. In Proceedings of the 15th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, pages 73–84, Rome, Italy, 2023. SCITEPRESS - Science and Technology Publications. [19] OpenAI. Openai agents sdk: Handoffs. https://openai.github.io/openai-agents-python/ handoffs/, 2025. Accessed: 2026-07-21. [20] Lorena Otero-Cerdeira, Francisco J Rodríguez-Martínez, and Alma Gómez-Rodríguez. Ontology matching: A literature review. Expert Systems with Applications, 42(2):949–971, 2015. [21] Samira Oulefki, Lamia Berkani, Nassim Boudjenah, Ladjel Bellatreche, and Aicha Mokhtari. BioGITOM: Matching Biomedical Ontologies with Graph Isomorphism Transformer. The VLDB Journal, 34(6):65, September 2025. [22] Yiping Song, Jiaoyan Chen, and Renate A Schmidt. Genom: ontology matching with description generation and large language models. World Wide Web, 29(3):29, 2026. [23] Steffen Staab and Rudi Studer. Handbook on ontologies. Springer Science & Business Media, 2013. [24] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 24824–24837, 2022. [25] An Yang et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024. [26] Hui Yang, Jiaoyan Chen, Yuan He, Yongsheng Gao, and Ian Horrocks. Language models as ontology encoders. In The Semantic Web – ISWC 2025: 24th International Semantic Web Conference, Nara, Japan, November 2–6, 2025, Proceedings, Part I, page 443–461, Berlin, Heidelberg, 2025. Springer-Verlag. 14
[27] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022.
A
Implementation Details
The benchmark is split into equivalence and subsumption evaluation subsets using a fixed random seed of 42 to ensure reproducibility. All open-source models, including the open-source backbone LLMs (Qwen2.5-32B-Instruct, Qwen2.5-72B-Instruct, Llama3.1-70B-Instruct, Mixtral-8x7BInstruct) and the local embedding, equivalence and subsumption baselines are run on two NVIDIA A100 GPUs.
B
Prompt Templates
B.1
Agent Initial Equivalence Screening Prompt
Agent A Prompt: Initial Equivalence Screening (System Message) Given one source concept and a list of candidate concepts. Task: Determine whether any candidate is the correct equivalent concept of the source. Equivalent means: • refers to the same real-world entity or concept • same meaning • same level of specificity If no exact equivalent exists, select NONE. Rules: • Select exactly one candidate ID, or NONE. • Do not invent candidate IDs. Return exactly this format: Reasoning: <short reasoning> Selected: <candidate_id or NONE> The source concept and candidate list are supplied via the user message at inference time.
B.2
Agent Equivalence Verification Prompt
Agent Prompt: Equivalence Verification (System Message) Given one source concept and a small list of candidate concepts. Task: Select the single best candidate as the final equivalence result. The correct result should: • have the same core meaning as the source • match the source granularity best Rules: • Always select exactly one candidate ID from the given list.
15
• Do not invent candidate IDs. Return exactly this format and no extra text: Reasoning: <short reasoning> Selected: <candidate_id> The source concept and the updated candidate set Cref = {ĉt , p∗ , ch∗ } are supplied via the user message at inference time.
B.3
Agent Subsumption Discovery Prompt
Agent Prompt (Iterative Search): Subsumption Discovery (System Message) Given one source concept and a list of candidate concepts. Task: Choose the nearest broader candidate for the source. If none is clearly broader, choose NONE. Rules: • A valid choice must be broader than the source. • It must be the closest available parent-level concept. • Do not choose candidates that are only related, part-based, sibling-level, or more specific. • Use only the given labels and synonyms. • Choose exactly one candidate ID, or NONE. Return exactly this format: Reasoning: <short reasoning> Selected: <candidate_id or NONE> The source concept and the current candidate set Ci are supplied via the user message at inference time. If NONE is returned, the candidate set is expanded to Ci+1 = P arents(Ci ) and this prompt is reapplied.
Agent Prompt (Fallback): Final Subsumer Selection (System Message) Given one source concept and a list of candidate concepts. Task: Select the candidate that is the closest broader concept of the source concept. Definitions: • A broader concept is a parent-level or ancestor-level concept that can subsume the source. • The selected candidate should be semantically broader than the source, not equivalent to it, not narrower than it, and not merely related to it. • If multiple candidates are broader, select the most specific and closest broader candidate. • You must select exactly one candidate from the list. Reasoning procedure: 1. Identify whether each plausible candidate is broader than the source. 2. Exclude candidates that are equivalent, narrower, sibling concepts, parts, attributes, or merely related
16
concepts. 3. Among the remaining broader candidates, choose the closest and most specific one. 4. If no candidate is a perfect direct parent, choose the best available broader candidate. Return exactly this format: Reasoning: <short reasoning> Selected: <candidate_id> This prompt is applied once, after dmax upward expansions, over the complete search trajectory C0 ∪C1 ∪· · ·∪Cdmax .
B.4
Conflict Resolution Prompt: Equivalence Arbitration
Conflict Resolution Prompt: Equivalence Arbitration (System Message) You are an ontology alignment arbitration agent. Your task is to choose which target concept is more appropriate as the equivalence match for the source concept. Choose exactly one option: A or B. Definitions: • equivalence means that the source concept and target concept refer to the same or nearly identical concept at the same conceptual granularity. Rules: • Use only labels and synonyms. • Do not use or infer from IRIs. • Do not introduce a new target concept. • Do not output subsumption. • Briefly explain the conflict. • Then output the final choice. Return exactly this format: Reason: <short reasoning> Choice: A or B This prompt is invoked when the lexical matching module and the agent-based reasoning module output two different equivalence target concepts (options A and B, supplied via the user message), following the third case of the conflict resolution rules in the Lexical Matching and Conflict Resolution section.
B.5
HOM Baseline Prompts
Direct Prompting Baseline (System Message) Given one source concept and a list of target candidate concepts. Task: Select exactly one candidate and assign exactly one relation: equivalence or subsumption. Definitions: • equivalence means the candidate refers to the same real world concept as the source. • subsumption means the candidate is broader than the source and can act as a direct or near direct
17
parent level concept. Rules: • Choose equivalence if one candidate has the same meaning as the source. • If no equivalent candidate exists, choose the nearest broader candidate. • Do not choose siblings, children, parts, or merely related concepts. • Prefer the most specific broader candidate when choosing subsumption. • You must choose exactly one candidate from the given list. • Use only the provided labels and synonyms. • Candidate IDs are only placeholders. They do not contain semantic information. Output JSON only: { "candidate_id": "...", "relation": "equivalence" or "subsumption" } The source concept and the candidate set are supplied via the user message at inference time. This prompt is used for both the LLM + C0 and LLM + Neighbourhood baselines, differing only in the candidate set provided.
Chain-of-Thought Prompting Baseline (System Message) Given one source concept and a list of target candidate concepts. Task: Select exactly one candidate and assign exactly one relation: equivalence or subsumption. Definitions: • equivalence means the candidate refers to the same real world concept as the source. • subsumption means the candidate is broader than the source and can act as a direct or near direct parent level concept. Rules: • Choose equivalence if one candidate has the same meaning as the source. • If no equivalent candidate exists, choose the nearest broader candidate. • Do not choose siblings, children, parts, or merely related concepts. • Prefer the most specific broader candidate when choosing subsumption. • You must choose exactly one candidate from the given list. • Use only the provided labels and synonyms. • Candidate IDs are only placeholders. They do not contain semantic information. • Keep the reasoning concise. • The reasoning should be natural language. • The final answer must be a JSON object. Output format: Reasoning: <concise natural-language reasoning> Final answer: { "candidate_id": "...",
18
Table 7: Performance of AgentMap under closed-source, hybrid, and open-source settings on the SNOMED–FMA Body task. Configuration
OverallAcc.
EqvAcc.
SubAcc.
Closed-source (OpenAI Emb. + GPT-4.1-mini)
0.657
0.957
0.354
Hybrid (OpenAI Emb. + Qwen2.5-32B-Instruct)
0.635
0.949
0.318
Hybrid (SBERT + GPT-4.1-mini)
0.545
0.777
0.312
Open-source (SBERT + Qwen2.5-32B-Instruct)
0.524
0.750
0.296
"relation": "equivalence" or "subsumption" } The source concept and the candidate set are supplied via the user message at inference time. This prompt is used for both the LLM + C0 + CoT and LLM + Neighbourhood + CoT baselines, differing only in the candidate set provided.
C
Effect of Open-Source Resources
C.1
Closed-Source vs. Open-Source Configurations
Table 7 reports results on SNOMED–FMA–Body under four configurations, combining OpenAI or SBERT embeddings with GPT-4.1-mini or Qwen2.5-32B-Instruct as the backbone. The fully closed-source configuration achieves the best performance (0.657 overall), while the fully opensource configuration (SBERT + Qwen2.5-32B-Instruct) trails by 13.3 points (0.524), a moderate gap given the removal of all commercial components. Comparing the two hybrid configurations suggests that the embedding model contributes more to this gap than the backbone LLM: replacing the backbone alone (OpenAI + Qwen2.5-32B-Instruct) costs only 2.2 points relative to the closedsource setting, whereas replacing the embedding model alone (SBERT + GPT-4.1-mini) costs 11.2 points, driven primarily by a large drop in equivalence accuracy (0.957 to 0.777). We examine this effect of embedding choice in more detail across all four benchmarks in the next subsection.
C.2
Effect of Embedding Model Across Benchmarks
While Table 7 controls for both factors on a single benchmark, we further isolate the effect of the embedding model alone by fixing the backbone to Qwen2.5-32B-Instruct across all four benchmarks. Table 8 reports the results. OpenAI embeddings outperform SBERT on both equivalence and subsumption accuracy on three of the four benchmarks (Body, Pharm, and Disease), indicating that retrieval quality generally affects both stages of AgentMap. However, which stage is more affected varies substantially across datasets: on SNOMED–FMA–Body, the embedding choice affects equivalence more (a 21.0% relative drop with SBERT) than subsumption (6.9 %), whereas on SNOMED–NCIT–Pharm the pattern reverses sharply, with subsumption accuracy dropping by over half (52.3%) while equivalence remains largely unaffected (0.6%). NCIT–DOID–Disease shows a more balanced effect on both metrics. This inconsistency suggests that the relative sensitivity of each stage to retrieval 19
Table 8: Effect of embedding model (OpenAI text-embedding-3-small vs. SBERT all-MiniLM-L6-v2) on HOM performance across the four benchmarks, with Qwen2.5-32B-Instruct fixed as the backbone LLM. Bold indicates the better result within each dataset. Dataset
Embedding Overall
Eqv
Sub
SNOMED–FMA–Body
OpenAI SBERT
0.635 0.524
0.949 0.318 0.750 0.296
SNOMED–NCIT–Pharm
OpenAI SBERT
0.528 0.374
0.978 0.386 0.972 0.184
NCIT–DOID–Disease
OpenAI SBERT
0.772 0.735
0.942 0.478 0.920 0.416
HeLiS–FoodOn
OpenAI SBERT
0.434 0.442
0.753 0.142 0.701 0.205
quality is dataset-dependent, though identifying the specific underlying factors is beyond the scope of this analysis. On HeLiS–FoodOn, the trend reverses entirely: SBERT achieves higher overall and subsumption accuracy than OpenAI embeddings, while OpenAI retains an advantage on equivalence. Given the small size of this dataset (174 equivalence and 190 subsumption instances), this reversal likely reflects sampling variance rather than a systematic advantage of SBERT. Overall, these results indicate that while AgentMap remains functional with a fully open-source embedding model, retrieval quality has a measurable and dataset-dependent impact on both equivalence and subsumption performance.
D
Effect of Candidate Set Size (Top-k)
We further examine the sensitivity of AgentMap to the size of the agent reasoning candidate set C0 , using Qwen2.5-32B-Instruct as the backbone, with k ∈ {5, 7, 10, 15}. Figure 2 reports the results across all four benchmarks. As k increases, Overall and Sub accuracy decrease monotonically, while Eqv accuracy rises only marginally. This indicates that a larger candidate set does not meaningfully improve equivalence discovery, since the correct equivalent concept is typically already covered by a small k, but it introduces more distractor candidates into Agent C’s subsumption search, increasing the likelihood of selecting an incorrect broader concept. This trend is consistent across the three biomedical benchmarks; HeLiS–FoodOn shows a similar pattern but with more fluctuation, reflecting its much smaller test set size. These results support our choice of k = 5 as the default configuration, which achieves the best overall and subsumption accuracy while minimising the LLM reasoning cost associated with a larger candidate set.
20
SNOMED-FMA-Body
1.0
0.8
Accuracy
Accuracy
1.0
SNOMED-NCIT-Pharm
Overall Eqv Sub
0.6 0.4
Overall Eqv Sub
0.8 0.6 0.4
0.2 5
7
10
0.2
15
5
7
Top-k
(a) SNOMED–FMA–Body NCIT-DOID-Disease 1.0
10
15
Top-k
(b) SNOMED–NCIT–Pharm HeliS-FoodOn
Overall Eqv Sub
1.0
Overall Eqv Sub
0.8
Accuracy
Accuracy
0.8
0.6 0.4 0.2
0.6 0.4 0.2
5
7
10
0.0
15
Top-k
5
7
10
15
Top-k
(c) NCIT–DOID–Disease
(d) HeLiS–FoodOn
Figure 2: Effect of candidate set size k on HOM results across the four benchmarks, using Qwen2.532B-Instruct as the backbone.
21