Single and Multi Truth Data Fusion using Large Language Models
arXiv:2606.28062v1 [cs.DB] 26 Jun 2026
Hira Beril Kucuk[0009−0002−4829−2156] , Norman W Paton[0000−0003−2008−6617] , Jiaoyan Chen[0000−0003−4643−6750] , and Zhenyu Wu[0000−0003−0981−5567] Department of Computer Science University of Manchester, Oxford Road, Manchester M13 9PL, UK {hiraberil.kucuk,norman.paton,jiaoyan.chen,zhenyu.wu}@manchester.ac.uk
Abstract. Data fusion, also known as truth discovery, is a data integration problem that aims to determine the correct value or set of values for each attribute of an object when presented with potentially conflicting values from multiple sources. Data fusion tasks belong to two main categories: single-truth scenarios, where each attribute has only one correct value, and multi-truth scenarios, where multiple values can be valid simultaneously. This paper investigates the use of Large Language Models (LLMs) in data fusion tasks for tabular data. Various prompting strategies, encompassing both single-truth and multi-truth scenarios, are investigated empirically. Domain-dependent, domain-independent, zeroshot and one-shot prompts are evaluated on three different benchmark datasets. Experimental results demonstrate that LLM-based approaches outperform traditional unsupervised truth discovery methods, such as DART and LTM, across all datasets. The codebase of this study has been made publicly available on GitHub1 . Keywords: Data Fusion, Truth Discovery, Large Language Model
1
Introduction
In recent years, the volume, variety and velocity of generated data has been rapidly increasing. This growth has made not only storing data but also obtaining meaningful and reliable information a significant research problem. In this context, data integration has become a prominent research area. Data integration can be characterised as including two main levels namely schema-level integration and instance-level integration [15][17]. Schema-level integration, as the initial stage of data integration, structurally and semantically aligns schemas from different sources and creates a common global schema; however, inconsistencies in the underlying data records may not be fully resolved at this stage. Instance-level data integration aims to integrate actual data records from different data sources. The primary goal of this stage is to identify records that represents the same real-world entity but have been recorded in different formats, incompletely, or inconsistently, and to merge them under a single, consistent representation. 1
https://github.com/hiraberil/LLM-DataFusion
2
H.B. Kucuk et al. Source 1
Source 2
Title: The Hobbit ISBN: 978-0547928227 Author: J.R.R. Tolkien Genre: Fantasy Publication Year: 1937 Language: English
Title: The Hobbit: There and Back Again ISBN: 9780547928227 Author: John Ronald Reuel Tolkien Genre: Fantasy; Adventure Publication Year: 1937 Language: en
Source 3
Fused record
Title: The Hobbit ISBN: 9780547928227 Author: Tolkien, J. R. R. Genre: Children’s literature; Fantasy Publication Year: 1937 Language: English
Title: The Hobbit ISBN: 9780547928227 Author: J.R.R. Tolkien Genre: {Fantasy, Adventure, Children’s literature} Publication Year: 1937 Language: English
Fig. 1. An example of data fusion for a book entity.
Data integration at the instance level typically involves the steps of format transformation, entity resolution and data fusion [15][20]. In an approach composed of these three steps in order, records are first standardised in format, then records representing the same real-world entity are identified, and finally, they are merged to create a single, consistent record [2][20]. Consider the following example: two different sources report the ISBN of a book as “ISBN 9780547928227 ” and “978-0547928227 ”. Format transformation may represent both using the format “9780547928227 ”, entity resolution will group different records containing equivalent ISBNs as representing the same book, and data fusion merges the book records into a single representation, as illustrated in Figure 1. Consider another example of fusion in the multi-truth setting, where multiple values of an attribute may be correct: three different sellers may report the genre of a book as being “Fantasy”, “Fantasy; Adventure”, and “Children’s literature; Fantasy”. The above data integration steps may output a record with all the valid genre values “Fantasy, Adventure, Children’s literature”, as illustrated in Figure 1. Data fusion methods face a number of challenges, particularly in multi-truth scenarios [2, 21]. A significant portion of existing methods make an assumption that attributes provide a single true value, and are therefore inadequate for modeling situations where multiple true values can exist [22]. While current unsupervised methods offer partial solutions using evidence from the data provided, they are mostly based on fixed assumptions and may fail to adequately account for context-sensitive semantic differences. Furthermore, the representation of the same real-world entity in different forms (e.g., “John Ronald Reuel Tolkien” and “J.R.R. Tolkien”) requires normalisation and semantic matching, a process not explicitly addressed in most traditional methods. LLMs are capable of modeling semantic relationships in natural language and making contextual inferences [23]. These capabilities offer potential advantages in addressing challenges such as implicit normalisation, expression diversity, and the interpretation of contradictory information encountered in data fusion. In particular, tasks such
Single and Multi Truth Data Fusion using Large Language Models
3
as comparing heterogeneous values from different sources at the semantic level and evaluating both single and multi-truth values together can be handled more flexibly with the context-based inference capabilities of LLMs. In this study, the data fusion problem is addressed as an LLM-supported data integration process. The main contributions of this paper are as follows: – An exploration of the use of LLMs for data fusion that includes zero-shot, single-shot, domain-independent and domain-dependent prompts for both single-truth and multi-truth settings. – An empirical evaluation in various benchmark scenarios in comparison with four classic data fusion methods. – An analysis of the strengths, limitations and reliability of LLM-supported data fusion. The remainder of this paper is structured as follows. Section 2 reviews related work on data fusion and LLM applications in data integration. Section 3 presents prompt-based strategies for data fusion and details the proposed methodology. Section 4 describes the experimental setup and evaluation criteria, and presents and analyses the results. Section 5 concludes the paper.
2
Related Work
2.1
Data Fusion
Data fusion is a critical step in instance-level data integration. It aims to reconcile potentially conflicting records from different sources, which are often obtained after schema matching, entity resolution, and format transformation, to achieve a more consistent and reliable record. Figure 1 illustrates three records of a book entity that need to be fused. For single-truth attributes such as ISBN, fusion is expected to output one correct value, whereas for multi-truth attributes such as author and genre, fusion is expected to output one or more correct values. Bleiholder and Naumann classify strategies for handling conflicting values into conflict-ignoring, conflict-avoiding and conflict-resolving approaches [2]. In practice, however, most research attention has focused on conflict-resolving methods, particularly truth discovery, which aims to infer the most credible value from multiple conflicting observations by jointly estimating source reliability and value correctness [7]. As summarised in more recent surveys, truth discovery methods are commonly characterised by how they model observations, source dependencies, and learning settings, and many unsupervised methods follow either a voting-style aggregation scheme or an iterative/probabilistic estimation process [11, 21]. Majority voting is one of the most straightforward approaches, which selects the value supported by the largest number of sources. More advanced truth discovery methods go beyond this strategy by jointly modelling factors such as source reliability and source dependence. Many such methods follow an iterative estimation paradigm, with detailed taxonomies and comparisons in recent surveys (e.g., [12, 21]) complementing descriptions of individual proposals (e.g., [5, 6, 11]).
4
H.B. Kucuk et al.
Here we briefly describe representative proposals that are used later as baselines in the experiments. The Latent Truth Model (LTM) adopts a probabilistic graphical model-based approach, treating the truth of each claimed value as a latent variable and jointly modelling it with source reliability [28]. It supports the multi-truth setting. Domain-Aware Truth Discovery (DART) extends classical bayesian approaches by considering expertise of sources in different domains [13]. While traditional methods generally assign a single reliability score to a source, DART calculates domain-based reliability scores for each source. This incorporates the fact that a source may be reliable in one domain but weak in another. These scores are used to calculate the probability of accuracy of the values.
2.2
LLMs in Data Integration
Recent studies demonstrate how LLMs can contribute to data integration, and proposals have been made that investigate how LLMs can support many different data integration tasks. These include schema matching [14], relationship discovery [8], entity resolution [27], format transformation [16] and schema inference [25]. As well as these proposals that focus on individual steps, several papers have explored how LLMs can contribute to data integration pipelines (e.g., [24, 18, 19]). These proposals may use embeddings to support semantic similarity or generative techniques to construct integrated representations at the schema or instance level. To the best of our knowledge, the following two studies represent the only existing LLM-based approaches that relate to data fusion. Both are part of multi-step pipelines. Ji et al. [9] discusses table integration in data lakes by dividing it into three sub-tasks: pairwise integrability judgment, integrable set discovery, and multi-tuple conflict resolution, where the latter step corresponds to data fusion. A single prompt is proposed that can be considered to be domaindependent, multi-shot and single-truth. This prompt is not compared with other prompts or baseline data fusion methods. Thus we complement and go further than this work in terms of data fusion by considering a wider range of prompt types, baselines and datasets. Steiner and Bizer [19] explore automating the end-to-end data integration process, encompassing schema matching, value normalization, entity matching, and data fusion, through LLMs. The proposal utilizes methods from an existing Python Data Integration library (PyDI), and LLMs play different roles at different stages in the pipeline. In terms of data fusion, the LLM is positioned as a tool that guides the configuration of existing methods rather than directly making the final data merging decision. Thus we complement this work by exploring a different role for the LLM in the data fusion process. Our work positions LLMs as a truth-discovery component that directly resolves conflicting values from multiple sources without relying on external tools, and evaluates this approach on widely used benchmark datasets.
Single and Multi Truth Data Fusion using Large Language Models
5
Fig. 2. Domain-Independent Examples – Multi-Truth Book & Movie Datasets. Domain-Dependent Examples are identical except that the Entity is referred to using its domain-specific type such as Book ISBN or Movie, and the sources are labelled as sellers in the Book dataset and as sources in the Movie dataset.
3
LLM Prompts for Data Fusion
The LLM prompts used in this study for data fusion can be classified into four types according to whether they are domain-independent (DI) or domaindependent (DD), and whether they are for single-truth (ST) or multi-truth (MT). These prompt categories are denoted as DI-ST, DI-MT, DD-ST and DD-MT. DI prompts can be applied to any domain without modification, regardless of the type of data to be combined; abstract terms such as entity, source and value are used. DD prompts, on the other hand, directly include terms associated with the target attribute (author, director, departure time, etc.) and need to be revised for a different domain. ST prompts are structured for scenarios where only one true value exists for an attribute (e.g., departure time or gate), while MT prompts are structured for scenarios where multiple valid values are possible (e.g., authors or directors). Book, movie and flights datasets are used in the experiments reported in Section 4. As the book and movie datasets used for evaluation in this study contain multi-valued attributes, the DI-MT and DD-MT prompts were applied to these datasets. Since the flight dataset consists of six single-valued attributes, the DD-ST and DI-ST categories were used for this dataset. Furthermore, each prompt was evaluated under both zero-sample (0-shot) and one-sample (1-shot) settings. In the 1-shot setting, a fixed input-output pair, as presented in Figures 2 and 3, is added to the beginning of the prompt; in the 0-shot setting, there is no such example block. The overall prompt structures are presented in Figures 4 and 5. Each prompt consists of an optional EXAMPLE block, a BODY block presenting the table data, a QUESTION block containing the task instructions, an optional CONSTRAINTS block containing additional constraints that modify the prompt’s behaviour, and a FORMAT block defining the expected output format. The prompts presented in these two figures omit the details of the CONSTRAINTS block; they are presented as follows. Prompt Constraints. Two constraints may be included in prompts, specifically:
6
H.B. Kucuk et al.
Fig. 3. Domain-Independent Examples – Single-Truth Flight Dataset. DomainDependent Examples are identical except that the Entity is referred to as Flight ID, and the generic attribute labels such as Attribute 1 are replaced by their domain-specific names such as Scheduled departure.
– C1: Only use values that appear in the sources above. This instructs the LLM not to invent values, but instead to choose from the values provided. – C2: If the same value appears in different formats, count them as one. This instructs the LLM to overlook representational inconsistencies.
4
Experimental Evaluation
4.1
Experimental Setup
Baselines. The following truth discovery methods are used as the baselines. (1) Majority Vote (MV) chooses the value with the highest frequency among all the input values for each attribute. This is a simple method without considering source reliability but involves the least computation. According to [7], its performance decreases in the multi-truth setting. (2) Source Reliability Vote (SRV) weights the voting process by iteratively estimating source reliability [10]. In this framework, values obtained from sources with higher reliability receive more weight; source reliability and value accuracy are updated iteratively. It can be regarded as a weighted extension of the basic voting approach. (3) LTM is a Bayesian framework that models source reliability and value accuracy as latent variables [28]. (4) DART is a domain-aware algorithm based on the observation
Single and Multi Truth Data Fusion using Large Language Models
7
Fig. 4. Design of Multi-Valued Prompt Structures for the Book Dataset. The prompt structure for the Movie dataset is identical, with the entity referred to as Movie: title - year instead of Book ISBN: isbn.
Fig. 5. Design of Single-Valued Prompt Structures for the Flight Dataset
8
H.B. Kucuk et al.
that the reliability of the same data source can differ significantly across different domains [13]. Domain expertise, value accuracy, and source reliability are updated concurrently in an iterative Bayesian framework. The implementations of these baselines are from an open-source Python repository for data fusion2 , and the default parameters from the original papers were used. Benchmark Datasets. The following criteria were considered in selecting the datasets: representation of different domains, coverage of both single- and multitruth settings, high density of inter-source discrepancies, and availability of the ground truth. Based on these criteria, the following three real-world datasets were used for evaluation, where two are multi-truth and one is single-truth. (1) The Book dataset is obtained from an open-source repository [26][5][4]. It contains author lists from AbeBooks.com, across 894 sources. The multi-valued nature of the author lists positions the dataset as a benchmark with multiple truths. (2) The Movie dataset is derived from an open-source data repository [1][13]. It contains director and year information for films compiled from 15 different web sources, including IMDb, Amazon and Metacritic. Its high levels of source contradiction and multi-accuracy structure provide a second basis for multi-truth comparison. (3) The Flight dataset is taken from the same opensource repository as the book dataset [4][11]. It includes departure and arrival information for over 1,200 flights compiled from 38 different sources. Since there is only one correct value for each object, it provides a single-truth scenario. Large Language Models (LLMs). Language understanding capacity, cost and API accessibility have been considered in selecting LLMs for evaluation. In the experiments, the GPT-4o-mini model was primarily preferred. It was observed that GPT-4o-mini has sufficient language understanding capacity for parsing tasks of this scale and was found to be suitable in terms of cost-performance balance. Unless otherwise specified, the results in this paper are based on GPT4o-mini. To evaluate the impact of model selection on performance, the prompt configuration that provided the highest performance for each dataset was also run with the GPT-4o and Claude Sonnet 4 models. For LLM runs, the temperature was set to 0 and the maximum number of tokens was set to 256. Evaluation Metrics. All the methods were evaluated using Recall, Precision, and F1-score, which are widely used in the data fusion literature [13] [7]. The definitions of true positive (TP), false negative (FN), and false positive (FP) differ in single-truth and multi-truth settings, as described below. Both Ground Truth and predicted values are subject to normalisation. Normalisation involves Unicode NFKD parsing, lowercase conversion, removal of titles from person names, and replacement of punctuation marks with spaces. Model performance is evaluated using the standard confusion matrix definitions. In the multi-truth experiments, each predicted value is independently evaluated against the set of true values, in accordance with the evaluation protocol 2
https://github.com/yishangru/TruthDiscovery
Single and Multi Truth Data Fusion using Large Language Models
9
proposed in [7]. For an entity, where T = t1 , t2 , . . . represents the set of true values and P = p1 , p2 , . . . represents the set of predicted values, a predicted value of pj is considered to match a true value of ti only when the word sets (after normalisation) are identical: match(ti , pj ) = 1 ⇐⇒ words(ti ) = words(pj ) Let P denote the predicted values, and T denote the values in the Ground Truth (GT). Under this formulation, a value is a true positive (TP) if it is predicted and matches a value in the GT (P ∩ T ), a false positive (FP) if it is predicted but does not match a value in the GT (P \ T ), and a false negative (FN) if it is in the GT but is not predicted (T \ P ). TP, FP and FN counts are then obtained for each fused record. The overall recall, precision and F1 scores are calculated based on the TP, FN and FP counts for each of records: P P TP 2·R·P TP P P , P =P , F1 = R= P TP + FN TP + FP R+P This definition allows classic single-valued baseline methods (DART, LTM, MV, SRV) to be evaluated within the same framework as LLM outputs across both single-valued and multi-valued datasets. 4.2
Results
In this section, experimental results are presented for each of the datasets in turn, comparing the baseline methods with DD and DI LLM prompts. Book The results on the Book dataset are presented in the left hand side of Table 1. The results in bold are those associated with the highest F1-score within a group of methods (baselines, domain-dependent and domain-independent). We can observe that the LLM with DD prompts demonstrate significantly better performance than the baselines and the LLM with DI prompts. The DD models produce reasonably balanced results in terms of recall and precision, and this is reflected in the F1 scores. The fact that the highest F1-scores are achieved in the DD-1shot, DD-C2-1shot, and DD-C1C2-1shot configurations demonstrates that the domain-dependent 1-shot prompts are the most effective on this dataset. In contrast, the performance of LLM with DI prompts is significantly lower; particularly in 0-shot scenarios, the low recall and precision values indicate that this approach fails to adapt sufficiently to the dataset. When examining the effect of example provision, it is observed that 1-shot generally improves performance over 0-shot. This improvement is particularly pronounced in DI prompts; that is, providing contextual examples offers a critical improvement for models with poor performance. In DD prompts, however, the already high performance increases in a more limited manner. Regarding the effects of the constraints C1 and C2, it is observed that C2 generally improves performance, whereas C1, when used alone, can sometimes reduce performance.
10
H.B. Kucuk et al.
However, the combined use of C1 and C2 is demonstrated to be more effective, particularly in the 1-shot scenario, where it is one of the configurations achieving the highest F1-scores. An analysis of the baseline methods indicates that the LTM method produces the most balanced results; however, these methods generally prioritise either recall or precision. For example, while DART produces high recall but low precision, SRV has very high precision but quite low recall. This imbalance negatively affects F1 scores. Overall, it is concluded that: (1) the DD prompt-based LLM offers more balanced precision and recall, and achieves higher performance compared to the baselines and DI prompt-based LLM; (2) example demonstration enhances LLM performance; and (3) the best result is obtained without the constraints C1 and/or C2. Movie The results on the Movie dataset are presented in the right hand side of Table 1. When examining the results for the movie dataset, it is observed that both baseline methods and LLM-based approaches exhibit higher and more balanced performance compared to the book dataset. Particularly noteworthy is the significantly better results produced by the LTM (F1=0.7959) and DART (F1=0.7768) methods compared to the book dataset. In contrast, the MV and SRV methods are insufficient in this dataset due to their single-value selection bias. In the examination of LLM-based methods, it is seen that DD prompts offer the highest and most balanced performance. DD-1shot yields the best result, while DD-C2 and DD-C1C2 variants exhibit similar performance. This demonstrates the strength of the DD prompts. DI prompts, however, have become more competitive in this dataset compared to the book dataset. In particular, the F1-scores of DI-0shot and DI-C1C2-0shot, which approach 0.78, indicate a significant improvement. While performance decreases with DI prompts compared to DD prompts, it appears that DI approaches can also perform well when sufficient semantic signals are provided. Indeed, the fact that the F1 difference between DD and DI methods in the movie dataset remains at a low level of approximately 0.03 shows that the model’s prior knowledge is similarly effective in both approaches in this dataset. Overall, the performance of both traditional baselines and the LLM-based methods is stronger in the movie dataset; however, the highest and most balanced results are still obtained by LLM with DD prompts. Nonetheless, it is clear that DI prompts have also become quite competitive. Flight The results on the Flight dataset are presented in the left hand side of Table 2. In the Flight dataset, baseline methods exhibit significantly higher performance compared to the other two datasets. The single-truth problem structure eliminates the need to model multi-truth sets, which has resulted in stronger performance for the single-truth baselines MV and SRV. However, the fact that
Single and Multi Truth Data Fusion using Large Language Models
11
Table 1. Experimental Results on Book and Movie Datasets. Note 1sh and 0sh denote 1-shot and 0-shot example demonstration, respectively. Method
Book Dataset Recall Precision
DART LTM MV SRV
Movie Dataset F1
Recall Precision
F1
0.8112 0.6782 0.4714 0.1463
0.4758 0.7137 0.7423 0.8570
0.5997 0.8776 0.6955 0.8209 0.5765 0.4687 0.2498 0.2537
0.6967 0.7725 0.7696 0.8763
0.7768 0.7959 0.5826 0.3935
DD-0shot 0.8339 DD-1shot 0.8102 DD-C1-0shot 0.8237 DD-C1-1shot 0.8051 DD-C2-0shot 0.8356 DD-C2-1shot 0.8237 DD-C1C2-0shot 0.8339 DD-C1C2-1shot 0.8305
0.7257 0.7551 0.6506 0.7121 0.7114 0.7341 0.6703 0.7335
0.7760 0.8777 0.7817 0.8690 0.7270 0.8603 0.7558 0.8210 0.7685 0.8603 0.7764 0.8428 0.7432 0.8734 0.7790 0.8384
0.7500 0.7713 0.7577 0.7673 0.7665 0.7814 0.7576 0.7773
0.8089 0.8172 0.8057 0.7932 0.8107 0.8109 0.8114 0.8067
DI-0shot DI-1shot DI-C1-0shot DI-C1-1shot DI-C2-0shot DI-C2-1shot DI-C1C2-0shot DI-C1C2-1shot
0.3212 0.4314 0.3099 0.3934 0.3407 0.5477 0.3258 0.4857
0.3792 0.8210 0.4520 0.7686 0.3760 0.8035 0.3983 0.7293 0.3960 0.8166 0.5310 0.7817 0.3829 0.8428 0.4878 0.7773
0.7373 0.7213 0.7244 0.7389 0.7276 0.7490 0.7366 0.7479
0.7769 0.7442 0.7619 0.7341 0.7695 0.7650 0.7862 0.7623
0.4627 0.4746 0.4780 0.4034 0.4729 0.5153 0.4644 0.4898
different sources report identical time information in different ways due to time zone differences, AM/PM representation, and the presence/absence of date prefixes creates a structural constraint for surface-level matching-based methods and determines the performance ceiling that traditional methods can achieve. One of the findings is that the DD and DI prompts exhibit similar performance (F1=0.9119 and F1=0.9118, respectively) in this dataset. With the DD prompts, the source constraint (C1) provides the highest DD performance with F1=0.9119. This constraint discourages the model from generating values not found in the sources, proving relevant for structured time data. In contrast, the addition of the format equivalence instruction (C2) reduces performance in both DD and DI prompts; this is most likely because the normalization function already resolves most format differences during the evaluation phase, and C2 creates an additional layer of complexity. With DI prompts, the best result was obtained with the C1C2, 1-shot configuration where all constraints were combined (F1=0.9118). A detailed look at the DD C1 0 Shot configuration, which yields the best F1 score, shows that results differed from attribute to attribute. The planned departure and planned arrival attributes achieved almost perfect accuracy (F1=0.9900 and 1.0000); sources generally agree on these values. In contrast, the actual de-
12
H.B. Kucuk et al. Table 2. Results on Flight Dataset under Flight ID and Obfuscated ID Settings Flight ID
Method
Obfuscated ID
Recall Precision
F1
Recall Precision
F1
DART LTM MV SRV
0.8487 0.7667 0.8653 0.7817 0.8506 0.7317 0.8653 0.7817
0.8056 0.8214 0.7867 0.8214
-
-
-
DD-0shot DD-1shot DD-C1-0shot DD-C1-1shot DD-C2-0shot DD-C2-1shot DD-C1C2-0shot DD-C1C2-1shot
0.9507 0.9489 0.9544 0.9398 0.9215 0.9361 0.9288 0.9398
0.8742 0.8725 0.8731 0.8641 0.8473 0.8593 0.8526 0.8612
0.9108 0.9091 0.9119 0.9003 0.8829 0.8961 0.8891 0.8988
0.9398 0.9453 0.9380 0.9288 0.9197 0.9325 0.9361 0.9380
0.8626 0.8677 0.8595 0.8540 0.8442 0.8559 0.8564 0.8595
0.8996 0.9048 0.8970 0.8899 0.8803 0.8926 0.8945 0.8970
DI-0shot 0.9380 0.8610 DI-1shot 0.9161 0.8423 DI-C1-0shot 0.9106 0.8358 DI-C1-1shot 0.9434 0.8660 DI-C2-0shot 0.8777 0.8070 DI-C2-1shot 0.9142 0.8392 DI-C1C2-0shot 0.9015 0.8261 DI-C1C2-1shot 0.9526 0.8744
0.8978 0.8776 0.8716 0.9031 0.8409 0.8751 0.8621 0.9118
0.8960 0.9124 0.8996 0.9252 0.8504 0.8942 0.8577 0.9288
0.8238 0.8375 0.8244 0.8478 0.7819 0.8208 0.7899 0.8555
0.8584 0.8734 0.8604 0.8848 0.8147 0.8559 0.8224 0.8906
parture (F1=0.8351) and actual arrival (F1=0.8854) showed lower performance. Error analysis reveals that a large portion of the failures in these attributes stemmed from small discrepancies in the relevant times.
Obfuscated Flight ID In the experiment results reported to date, it is not obvious to what extent the LLM approaches are using background knowledge to select values for inclusion in the fused result. Information on books, movies and flights is publicly available, and can be assumed to be used during LLM training. Experiments carried out using LLMs on public datasets are open to the criticism that the results may have been significantly different in domains that are less well represented in public data [3]. To reduce the ability of the LLM to use background knowledge to fuse flight data, we have repeated the experiments on the flight dataset with an obfuscated Flight ID. Thus, instead of a Flight Id of the form Airline-FlightCode-From-To (e.g., AA-1007-MIA-PHX ), an Id of the form FLIGHT-Integer (e.g., FLIGHT001 ) has been used, making it difficult for the LLM to use background timetable knowledge during data fusion. This change was applied only to the LLM-based prompts, because truth discovery methods such as DART, LTM, MV and SRV
Single and Multi Truth Data Fusion using Large Language Models
13
Table 3. Comparison of LLM performance across datasets. Results report the bestperforming prompt configuration per dataset for each model. Dataset
GPT-4o-mini Recall Precision
Book Movie Flight
GPT-4o F1
Recall Precision
Claude Sonnet 4.6 F1
Recall Precision
F1
0.8102 0.7551 0.7817 0.7983 0.7572 0.7772 0.8254 0.7718 0.7977 0.8690 0.7713 0.8172 0.7118 0.7689 0.7392 0.7991 0.8097 0.8044 0.9544 0.8731 0.9119 0.8467 0.7798 0.8119 0.2153 0.7662 0.3362
act only on the values from the column to be fused, and are therefore immune to changes in FlightId. The results with the obfuscated Flight Id are presented in the right hand side of Table 2. The change in the FlightId format had a small impact on the F1 score; the best result decreased only slightly, from 0.9119 to 0.9048. Across the range of results, the F1 score dropped in 8 out of 9 DD cases and in 9 out of 9 DI cases, with an overall average reduction of 0.014. This small drop indicates that the LLM derives limited benefit from the semantic information encoded in the original flight IDs. Overall, together with the close performance of DD and DI prompts, this suggests that the model’s decisions are driven mainly by the consistency patterns among source claims rather than by domain-specific cues in the entity key. LLM Selection The results for different LLMs are shown in Table 3. Following on from the results obtained with the GPT-4o-mini model, the prompt combinations that yielded the best F1 score were also tested with the GPT-4o and Claude Sonnet 4.6 models. In this approach, whose primary aim was to examine the impact of model selection on performance, Claude Sonnet provided the best F1 score on the Book dataset, but much the worst recall and F1 score with the single-valued prompt on the Flight dataset. The best results were obtained with the GPT-4o-mini in the Movie and Flight datasets. Costs Table 4 presents the costs incurred carrying out data fusion on the benchmark datasets, with the associated dataset sizes. Whether or not these are prohibitive would depend on the application and the user. The elapsed time per API call for DD-1shot averaged 1.35s/request for Book, 1.14s/request for Movie and 3.28s/request for Flight. Table 4. Financial cost for GPT-4o-mini fusing the benchmark datasets Dataset Cost Input Tokens Output Tokens Total Tokens Book Movie Flight
$0.57 $0.15 $0.48
3.615M 0.762M 2.811M
73,448 26,715 127,181
3.688M 0.789M 2.938M
14
H.B. Kucuk et al.
Summary When the results obtained across the benchmark datasets are evaluated as a whole, a number of observations can be made: 1. DD LLM-based approaches consistently produce more balanced precision/recall results and higher F1 scores than the baseline methods. 2. DD prompts provided better F1 scores than the corresponding DI prompts in 8 out of 8 cases in the Book dataset, 8 out of 8 cases in the Movie dataset, and 6 out of 8 cases in the Flights dataset, so we can say that domain-specific wording in prompts is useful for data fusion. We note that the domain-specific wording used should be able to be generated, rather than hand-crafted, in many cases. 3. The 1-shot prompts outperformed the 0-shot prompts in F1 score in 8 out of 8 cases for Books, 2 out of 8 cases for Movies, and 5 out of 8 cases for Flights, so the impact of examples is inconsistent across the datasets. 4. The C1 constraint improved the F1 score compared with no constraint in 0 out of 4 cases for the Book dataset, 0 out of 4 cases for the Movie dataset, and 2 out of 4 cases for the Flight dataset. The C2 constraint improved the F1 score compared with no constraint in 2 out of 4 cases for the Book dataset, 2 out of 4 cases for the Movie dataset, and 0 out of 4 cases for the Flight dataset. The C1+C2 constraints improved the F1 score compared with no constraint in 2 out of 4 cases for the Book dataset, 3 out of 4 cases for the Movie dataset, and 1 out of 4 cases for the Flight dataset. As such, the experiments do not provide compelling evidence for including the constraints. 5. Obfuscating the Flight ID in the Flights dataset led to a small but consistent reduction in performance. In 15 out of 16 cases, the LLM prompts with the obfuscated Flight ID provided a higher F1 score than the best performing baseline.
5
Conclusions
Large language models are having a substantial impact on many data intensive tasks. To the best of our knowledge, this paper provides the first wide-ranging proposal for, and evaluation of, prompt-based approaches to data fusion. Here we revisit the claimed contributions of the paper from the introduction to highlight the nature of the contributions made: 1. An exploration of the use of LLMs for data fusion that includes zero-shot, single-shot, domain-independent and domain-dependent prompts for both singletruth and multi-truth settings. Separate prompts have been proposed for single and multi-truth settings, and in both such settings 0-shot and 1-shot, domain-dependent and domain-independent variants have been derived. The addition of constraints to guide the LLM in how to perform data fusion has also been investigated. 2. An empirical evaluation in various benchmark scenarios in comparison with four classic data fusion methods. The various prompts from (1) have been
Single and Multi Truth Data Fusion using Large Language Models
15
compared with 4 (also unsupervised) baseline proposals on 3 benchmark datasets. The baselines include majority voting and three high profile techniques published in SIGMOD and PVLDB that between them have well over 1000 Google Scholar citations. The domain-dependent LLM-based proposals consistently outperformed all the baselines on all the datasets. 3. An analysis of the strengths, limitations and reliability of LLM-supported data fusion. The wide-ranging evaluation provides various insights into prompt design for data fusion. Including domain-specific terminology in prompts sometimes provided significant benefits, but the provision of examples and constraints provided fewer and less consistent benefits. In an experiment designed to reduce the extent to which background knowledge could be exploited by the LLMs, LLM-based approaches still outperformed the baselines. Acknowledgments. Hira Beril Kucuk’s PhD research is supported by the Ministry of National Education of the Republic of Türkiye, and sincere thanks are extended for their generous support. Disclosure of Interests. The authors declare that they have no competing interests.
References 1. heathersherry.github.io personal website. https://heathersherry.github.io/ (2026), accessed: 2026-03-30 2. Bleiholder, J., Naumann, F.: Data Fusion. ACM Computing Surveys 41(1), 1–41 (1 2009) 3. Bodensohn, J., Brackmann, U., Vogel, L., Sanghi, A., Binnig, C.: Unveiling challenges for llms in enterprise data engineering. Proc. VLDB Endow. 19(2), 196–209 (2025), https://www.vldb.org/pvldb/vol19/p196-bodensohn.pdf 4. Dong, X.L.: Fusion datasets. https://lunadong.com/fusiondatasets (2026), accessed: 2026-03-30 5. Dong, X.L., Berti-Equille, L., Srivastava, D.: Integrating conflicting data: the role of source dependence. Proc. VLDB Endow. 2(1), 550–561 (Aug 2009) 6. Dong, X.L., Naumann, F.: Data fusion: resolving data conflicts for integration. Proc. VLDB Endow. 2(2), 1654–1655 (Aug 2009) 7. Dong, X.L., Saha, B., Srivastava, D.: Less is more: selecting sources wisely for integration. Proc. VLDB Endow. 6(2), 37–48 (Dec 2012) 8. Dong, Y., Xiao, C., Nozawa, T., Enomoto, M., Oyamada, M.: Deepjoin: Joinable table discovery with pre-trained language models. Proc. VLDB Endow. 16(10), 2458–2470 (2023). https://doi.org/10.14778/3603581.3603587 9. Ji, D., Luo, H., Bao, Z., Culpepper, J.S.: Table integration in data lakes unleashed: pairwise integrability judgment, integrable set discovery, and multi-tuple conflict resolution. The VLDB Journal 34, 36 (2025) 10. Li, Q., Li, Y., Gao, J., Zhao, B., Fan, W., Han, J.: Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation. Proc. ACM SIGMOD pp. 1187–1198 (2014) 11. Li, X., Dong, X.L., Lyons, K., Meng, W., Srivastava, D.: Truth finding on the deep web: is the problem solved? Proc. VLDB Endow. 6(2), 97–108 (Dec 2012)
16
H.B. Kucuk et al.
12. Li, Y., Gao, J., Meng, C., Li, Q., Su, L., Zhao, B., Fan, W., Han, J.: A survey on truth discovery. SIGKDD Explor. Newsl. 17(2), 1–16 (Feb 2016) 13. Lin, X., Chen, L.: Domain-aware multi-truth discovery from conflicting sources. Proc. VLDB Endow. 11(5), 635–647 (Jan 2018) 14. Liu, Y., Peña, E., Santos, A.S.R., Wu, E., Freire, J.: Magneto: Combining small and large language models for schema matching. Proc. VLDB Endow. 18(8), 2681–2694 (2025). https://doi.org/10.14778/3742728.3742757 15. Mandreoli, F., Montangero, M.: Dealing With Data Heterogeneity in a Data Fusion Perspective: Models, Methodologies, and Algorithms. Data Handling in Science and Technology 31, 235–270 (1 2019) 16. Nobari, A.D., Rafiei, D.: Tabulax: Leveraging large language models for multiclass table transformations. Proc. VLDB Endow. 18(11), 3826–3839 (2025). https://doi.org/10.14778/3749646.3749657 17. Putrama, I.M., Martinek, P.: Heterogeneous data integration: Challenges and opportunities. Data in Brief 56, 110853 (10 2024) 18. Qian, Y., et al.: Unidm: A unified framework for data manipulation with large language models. In: Proceedings of the Seventh Annual Conference on Machine Learning and Systems, MLSys 2024 (2024) 19. Steiner, A., Bizer, C.: Automatic end-to-end data integration using large language models. CoRR abs/2603.10547 (2026). https://doi.org/10.48550/ARXIV.2603.10547 20. Tian, Y., Sun, L., Potdar, V., Song, B., Koukaras, P.: Data Integration and Storage Strategies in Heterogeneous Analytical Systems: Architectures, Methods, and Interoperability Challenges. Information 2025, Vol. 16, Page 932 16(11), 932 (10 2025) 21. Wang, S., Zhang, H., Sheng, Q.Z., Li, X., Sun, Z., Cai, T., Zhang, W.E., Yang, J., Gao, Q.: A survey on truth discovery: Concepts, methods, applications, and opportunities. IEEE Trans. Big Data 11(2), 314–332 (2025) 22. Wang, X., Sheng, Q.Z., Yao, L., Li, X., Fang, X.S., Xu, X., Benatallah, B.: Empowering truth discovery with multi-truth prediction. In: Proc. 25th ACM CIKM. p. 881–890. CIKM ’16 (2016) 23. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824–24837 (2022) 24. Wrembel, R.: Data integration in the ai era: Research trends and still open issues. In: Leung, C.K., Dignös, A., Kotsis, G., Tjoa, A.M., Khalil, I. (eds.) Big Data Analytics and Knowledge Discovery. pp. 21–36 (2026) 25. Wu, Z., Chen, J., Paton, N.W.: Taxonomy inference for tabular data using large language models. In: The Semantic Web - 22nd European Semantic Web Conference, ESWC, Part I. pp. 403–422 (2025). https://doi.org/10.1007/978-3-031-945755\_22 26. Yin, X., Han, J., Yu, P.S.: Truth discovery with multiple conflicting information providers on the web. In: Proc. 13th ACM SIGKDD. p. 1048–1052. KDD ’07, ACM (2007) 27. Zeakis, A., Papadakis, G., Skoutas, D., Koubarakis, M.: An in-depth analysis of pre-trained embeddings for entity resolution. VLDB J. 34(1), 5 (2025). https://doi.org/10.1007/S00778-024-00879-4 28. Zhao, B., Rubinstein, B.I.P., Gemmell, J., Han, J.: A bayesian approach to discovering truth from conflicting sources for data integration. Proc. VLDB Endow. 5(6), 550–561 (Feb 2012)