Precision Grounding: augmenting large language models with evidence-based databases for trustworthy genetic variant summarization.

Precision Grounding: Augmenting Large Language Models with Evidence-Based Databases for Trustworthy Genetic Variant Summarization - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice Int J Med Inform . Author manuscript; available in PMC: 2026 May 31. Published in final edited form as: Int J Med Inform. 2026 Apr 2;214:106424. doi: 10.1016/j.ijmedinf.2026.106424 Precision Grounding: Augmenting Large Language Models with Evidence-Based Databases for Trustworthy Genetic Variant Summarization Xinsong Du 1, 2, *, # , Anna Nagy 3, *, # , Michael F Oates 3, 4 , Yifei Wang 5 , Xinyi Wang 1, 2 , Joseph M Plasek 1, 2 , Samuel J Aronson 3, 4 , Matthew S Lebo 3, 6, 7 , Li Zhou 1, 2 Author information Article notes Copyright and License information 1 Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, United States 2 Department of Biomedical Informatics, Harvard Medical School, Boston, United States 3 Laboratory for Molecular Medicine, Mass General Brigham, Boston, United States 4 Digital Research Operations, Mass General Brigham, Boston, United States 5 Michtom School of Computer Science, Brandeis University, Waltham, United States 6 Department of Pathology, Brigham and Women’s Hospital and Harvard Medical School, Boston, United States 7 Molecular and Population Genetics, Broad Institute of Harvard and MIT, Boston, United States * Equal Contribution Author Contributions Conceptualization and design: XD, AN, MFO, SJA, MSL, LZ. Programming and software development: XD, MFO. Prompt design for large language models: XD, AN, XW, LZ. Large language model generated response preparation: XD, XW. Gold standard variant summary preparation: AN. Genetic variant summary evaluation guideline preparation and manual evaluation: AN, MSL. Calculation of correlation between natural language processing metrics and expert evaluation results: YW. Results interpretation: XD, AN, MFO, JMP, YW, SJA, LZ. Original draft preparation: XD, AN. Significant contribution to paper revision: XD, AN, SJA, LZ. Funding Acquisition: XD, MSL, LZ. Supervision: LZ. ✉ Corresponding author: Xinsong Du, PhD, Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145, xidu1@bwh.harvard.edu # Contributed equally. Issue date 2026 Jul 1. PMC Copyright notice PMCID: PMC13222032  NIHMSID: NIHMS2163911  PMID: 41950627 The publisher's version of this article is available at Int J Med Inform Previous version available: This article is based on a previously available preprint posted on medRxiv on June 10, 2025: " Precision Grounding: Augmenting Large Language Models with Evidence-Based Databases for Trustworthy Genetic Variant Summarization ". Abstract Purpose: To propose a novel method that augments LLMs with evidence-based, variant-specific information to improve summarization accuracy and support clinical decision making. Methods: We proposed Precision Grounding which uses a query tool that integrates domain expert-selected resources and allows users to query relevant factual information using identifiers in databases. In our case, which is genetic variant summarization, we developed CATT (ClinGen website: https://shorturl.at/pw81X ; GitHub: https://github.com/mgbpm/clingen-ai-tools ; Zenodo: https://doi.org/10.5281/zenodo.18896080 ), an open-source tool integrating publicly available ClinGen, ClinVar, and GenCC databases. Users can query and retrieve curated evidence via Variation IDs to ground LLM outputs. We compared our approach with baseline methods, including web-search grounding and retrieval-augmented generation (RAG; specifically, MedRAG) using 50 expert-selected variants. Results: GPT-4o was selected due to its good performance on our task during a pilot test. Using GPT-4o, we found web-search grounding performed better than MedRAG since MedRAG failed to generate clinically useful summaries due to limitations of relevant information in its databases. Precision Grounding outperformed web-search grounding, achieving significantly higher accuracy and completeness scores, which were based on a 5-point Likert-Scale of 4.76 (+0.74) and 4.94 (+0.84), respectively. Error analysis revealed that precision grounding reduced clinically significant hallucinations, such as incorrect pathogenicity classification and summarizing the wrong variant. Conclusion: Precision Grounding outperformed existing grounding approaches for genetic variant summarization. Our open-source tool, CATT, enables integration of curated, domain-specific genetic variant knowledge and significantly reduces hallucinations in LLM outputs. By enhancing the accuracy and completeness of variant interpretation, this framework holds strong potential for real-world implementation in clinical decision-support systems. Its modular, interoperable design allows for seamless integration into clinical genomics workflows, supporting more efficient, trustworthy, and scalable variant review processes. Keywords: Clinical Decision Support, Large Language Models, Genomics, Hallucination, Summarization, Retrieval Augmented Generation 1. INTRODUCTION Genomic variation plays a significant role in human disease, with approximately 5 million variants present in each individual’s genome. 1 – 3 Technological advances have made it routine to detect these variants, fueling progress in both basic and clinical research. 4 However, the ability to detect DNA variation has outpaced our capacity to interpret the clinical relevance of these variants. 5 Current practices focus on only a small subset of variants, limiting the potential impact of genetic testing and its contribution to research. 1 This gap is partly due to the high personnel costs required for the in-depth assessment and summarization of evidence related to specific genes and variants. 6 Fortunately, large language models (LLMs) hold significant potential to enhance clinical practice by automating the interpretation and summarization of genetic variants, thereby reducing costs and improving accessibility to genetic insights. However, LLMs are prone to generating inaccurate or misleading information—a phenomenon known as hallucination. 7 , 8 Web-search grounding and retrieval augmented generation (RAG) using vector databases have been widely used by existing LLMs such as ChatGPT to mitigate hallucinations. Web-search grounding retrieves information from the internet using services like Bing Search. RAG retrieves information by extracting semantically relevant documents from a pre-indexed corpus using vector embeddings and similarity search and then incorporates the retrieved content into the model’s context to support more accurate and evidence-based responses. 9 However, web-search grounding and general RAG systems may not consistently provide high-quality or clinically validated contextual information because they rely on uncurated web sources or semantically retrieved documents. As a result, the retrieved evidence may be of low quality or limited relevance, reducing the reliability of these approaches for clinical applications. 10 – 13 In genomic medicine, such hallucinations will limit the ways these tools can be used, as incorrect variant interpretations may lead to incorrect pathogenicity classifications, flawed clinical summaries, and inconsistent evidence representation. These errors can have serious consequences in clinical decision-making, potentially leading to unnecessary medical interventions, misdiagnoses, or missed opportunities for timely treatment. To address these challenges, we propose a novel and user-friendly method called Precision Grounding that enhances LLM with publicly available domain-specific databases. Unlike Web-Search Grounding or traditional RAG, Precision Grounding uses a query tool which allows users to query relevant information based on identifiers in publicly available databases. In the context of generic variant summarization, we developed CATT (ClinGen AI data Transformation Tool) as the query tool. Notably, CATT harmonized ClinGen, 1 ClinVar, 8 and GenCC 9 as resources for retrieval augmentation and allows users to query factual information using Variation IDs in ClinVar to ground LLMs. By implementing Precision Grounding, we aim to improve the factual accuracy, consistency, and clinical relevance of LLM-generated interpretations. This work fills a critical gap in applying LLMs to genomic medicine and supports safer, more reliable, and evidence-informed use of generative models in clinical decision support. 2. METHODS 2.1. CATT We developed CATT to integrate and harmonize expert curated genetic databases, including ClinGen, ClinVar, and GenCC, which are developed to support the understanding of relationships between human genetic variation and health. CATT is open-source and publicly accessible via the ClinGen website ( https://shorturl.at/pw81X ), GitHub ( https://github.com/mgbpm/clingen-ai-tools ), and Zenodo ( https://doi.org/10.5281/zenodo.18896080 ). Data from these sources are linked using standardized identifiers such as HGNC gene IDs and ClinVar Variation IDs to ensure consistency across datasets. During harmonization, variants are normalized using standardized identifiers and genomic representations, and records referring to the same variant across databases are grouped to remove duplicate entries while preserving source-specific annotations for traceability. The harmonization pipeline retrieves publicly available database releases, and the first step “downloads of all sources” can be rerun to incorporate updates when new database releases become available. When conflicting annotations occur across databases, CATT retains all source annotations rather than resolving discrepancies, allowing downstream users or LLM systems to consider the full evidence context. Data quality is supported through the use of expert-curated source databases, automated validation of identifiers and variant fields during preprocessing, and preservation of source attribution for each integrated record. Users can query the integrated database by specifying variant identifiers via the command line. CATT outputs queried genetic and clinical data in formats optimized for LLMs and other AI algorithms. The tool generates two primary types of output: template-based natural language text and structured CSV files, both designed for direct use in LLM and AI workflows. This dual-output design enables flexible incorporation of evidence-based knowledge into prompt engineering or downstream AI applications, potentially enabling more robust variant interpretation and clinical decision support. Notably, CATT’s modular architecture supports future extensibility; new data sources can be incorporated by adding a subdirectory to the “sources” folder, with metadata and download specifications provided through a standardized set of files (“config.yaml”, “dictionary.csv”, and “mapping.csv”). Additionally, CATT supports batch processing, allowing querying multiple variants in a single run. 2.1.1. Template-Based Text Generation The tool converts structured genetic and clinical data into natural language descriptions using Genshi Template ( https://genshi.readthedocs.io/en/latest/upgrade.html#text-templates ), which is a flexible, standardized, and well-maintained tool. This approach allows for the dynamic generation of text summaries that preserve critical contextual and clinical details. Each dataset field, such as gene name, variant classification, or clinical significance, is mapped to a corresponding narrative structure. Users can define custom templates to specify how these structured data points should be expressed in natural language. During processing, the tool applies the predefined templates to individual records, generating text outputs. The templates incorporate conditional phrasing, allowing the generated text to adapt based on field values. This provides context for the content, particularly for categorical and numerical data. The final text representations can be saved as separate text files for each record or concatenated into a single document. 2.1.2. CSV-Based Data Representation In addition to generating templated-based text files, CATT provides an option to structure genetic variant data into CSV format. This structured format allows LLMs to analyze tabular genetic information in conjunction with natural language queries, particularly in attachment-based input workflows where LLMs process structured data as supplementary input. The tool allows users to specify which gene- and variant-level features should be included in the CSV output, such as gene symbol, variant ID, clinical significance, and pathogenicity classification. The column formatting is standardized to ensure consistency across records, making the data easily interpretable by LLMs. Categorical data can be converted into human-readable text to facilitate structured reasoning without requiring additional preprocessing. The CSV output can be used directly as an attachment for LLM queries, enabling models to extract insights from structured clinical and genomic records. 2.2. Precision Grounding to Augment LLMs to Increase Variant Summarization Trustworthiness We define the trustworthiness of variant summarization as the extent to which LLM-generated outputs are accurate, complete, and grounded in authoritative genomic databases, thereby minimizing hallucinations and ensuring that the summarized information reliably reflects current domain knowledge. 14 – 16 To ground LLMs and improve trustworthiness of variant summarization, Precision Grounding retrieves relevant information through a query tool which grabs information from databases using domain-specific unique identifiers. We developed open-source tool CATT to support Precision Grounding in genomics domain. CATT allows users to query information from integrated genomics databases ( Figure 1 ). Specifically, our pipeline includes three steps: Step 1: Query CATT for user specified genetic variant information and get retrieved factual information in .txt and .csv files. Example .txt and .csv files are available in our GitHub repository ( https://shorturl.at/tSQvK ) and Zenodo. 17 Step 2: Upload these files as attachments into the LLM interface. Step 3: Prompt the LLM to generate a clinically useful variant summary. Figure 1. Precision Grounding vs Traditional Methods for Augmenting LLMs in Genetic Variant Summarization. Open in a new tab Traditional methods include web-search grounding and retrieval augmented generation (e.g., MedRAG) through vector databases. Traditional methods may not consistently provide high-quality or clinically validated contextual information because they rely on uncurated web sources or semantically retrieved documents. As a result, the retrieved evidence may be of low quality or limited relevance, reducing the reliability of these approaches for clinical applications. Our Precision Grounding approach obtains factual information through querying domain-specific databases using unique identifiers. It involves three steps: (1) generating curated variant-specific information in .txt and .csv formats using the CATT tool, (2) attaching these files to the LLM as background context, and (3) prompting the LLM to summarize the variant based on the grounded evidence. 2.3. Evaluation of The Effectiveness of Precision Grounding The evaluation consisted of three phases: 1) LLM selection, 2) evaluation method selection, and 3) comparison and evaluation of precision grounding and baseline grounding methods. During the LLM selection phase, we aimed to identify the most accurate LLM for genetic variant summarization when precision grounding was not employed. Additionally, our recent systematic review shows that a wide range of evaluation approaches have been used to assess LLM performance. 18 Therefore, we conducted a second phase to determine the most appropriate evaluation method for this task. Manual expert evaluation remains the gold standard because it captures clinical correctness and contextual relevance; however, it is resource-intensive, time-consuming, and difficult to scale. In contrast, automated NLP metrics are scalable but often show limited correlation with expert assessments in clinical applications. Accordingly, we incorporated manual evaluation using a Likert-scale scoring framework, which has been widely used to assess dimensions such as accuracy and completeness in summarization tasks. 18 In addition, we evaluated several automated NLP metrics. Our recent systematic review suggests that metrics such as ROUGE, BLEU, METEOR, and cosine similarity between embedding representations of generated and reference texts (e.g., embeddings generated by transformer-based models such as BERT or Llama) may show relatively higher correlation with human evaluation compared with other automated metrics. 18 Therefore, these methods were included in our evaluation. We also tested AlignScore, a recently proposed metric designed to assess factual consistency between generated and reference texts. The purpose of testing these automated metrics was to determine whether any of them could reliably approximate expert judgment and thus enable scalable evaluation of LLM-generated variant summaries. If no automated metric demonstrated sufficient alignment with expert evaluation, manual assessment would remain the primary evaluation approach to ensure reliability. In the final phase, we assessed the performance of precision grounding using the selected LLM and evaluation method. We also evaluated a baseline approach, web-search grounding, which is integrated into many LLM systems and widely used in practice. The performance of precision grounding was then compared with the baseline method. A total of 50 representative genetic variants were selected for the purpose of testing and evaluation, with 10 variants used in the first and second phases and all 50 variants used in the third phase. Variant selection was designed to capture the diversity of existing variant spectrum, considering factors such as variant type (e.g., loss of function, missense, splicing), ClinVar pathogenicity classification (e.g., benign, pathogenic, risk allele), submitter concordance or discordance in ClinVar, and information density (measured by the number of gene and variant classification submissions). 2.3.1. LLM Selection We first identified the most accurate LLM for genetic variant summarization by evaluating four widely used LLMs, including GPT-4o, o1, Claude 3.5 Sonnet, and DeepSeek R1. Because the intended application of precision grounding is to support clinical decision making, model performance, particularly the accuracy and completeness of variant summaries, it was an important consideration when selecting candidate models. At the time of this study, these models were among the most widely used high-performing systems for summarization tasks. In contrast, many open-source models (e.g., LLaMA) often require substantial local computational resources for deployment and were less commonly used in clinical research settings at the time of our evaluation. 18 Therefore, we focused on these widely adopted high-performance models while designing the proposed Precision Grounding framework to remain model-agnostic so that it can be readily applied to both proprietary and open-source LLMs in future work. The prompts to LLMs used for variant summarization were standardized across all evaluated LLMs to ensure fair comparison. Prompt design was iteratively refined through pilot testing and discussions among study authors (XD, AN, HW). As illustrated in Supplementary Figure S1 , the final prompt included three key components. First, both the genetic variant name and the corresponding identifier (e.g., ClinVar Variation ID) were provided, as we observed that the LLM would fabricate name or ID without specifying both in the prompt. Second, a single example variant summary written by genetic counselor (AN) was included in the prompt to guide the model toward producing responses in a consistent and clinically organized format; preliminary testing indicated that adding more examples did not meaningfully improve output quality. Third, the prompt explicitly instructed the LLM to generate a comprehensive summary of the specified variant that would be useful for clinicians, encouraging the inclusion of clinically relevant information such as pathogenicity classification, gene–disease relationships, and supporting evidence from curated databases. 2.3.2. Evaluation Method Selection 2.3.2.1. Human Evaluation We evaluated the accuracy and completeness since they are the most commonly-used evaluation for LLM responses, 19 and they are particularly critical in the context of genetic variant summarization where clinical decision making is at stake. The evaluation rubric was developed via a discussion between our certified genetic counselor (AN) and her mentor who was a genomic medicine expert (MSL). During the early phase of the study, the rubric and example evaluations were presented and discussed within the research team by AN during regular meetings to ensure clarity and consistency in the evaluation criteria. After establishing the rubric, AN independently evaluated all variant summaries for accuracy and completeness using the predefined Likert-scale scoring system and documented specific errors identified in each LLM-generated summary. To minimize subjective bias, LLMs’ responses were anonymized and randomized during evaluation, ensuring the reviewers were blinded to the identity of the model that generated each response. This approach allowed consistent expert-driven evaluation while maintaining feasibility given the specialized expertise required for genetic variant interpretation. Accuracy was manually evaluated using a 5-point Likert-Scale, and a higher accuracy means less clinically harmful. 20 A score of 1 indicated very poor accuracy, characterized by major errors that could lead to harmful interpretation of the variant’s information. A score of 2 reflected poor accuracy with notable errors impacting understanding. A score of 3 corresponded to moderate accuracy, with inaccuracies primarily affecting supporting details rather than key findings. A score of 4 indicated minor inaccuracies or hallucinations that did not significantly alter interpretation. A score of 5 denoted perfect accuracy, devoid of any errors. Completeness was similarly assessed using a 5-point Likert-Scale. 20 A score of 1 indicated a complete absence of relevant information from key sources (ClinVar, ClinGen, or GenCC). A score of 2 reflected significant omissions that impaired the interpretation of the variant and/or associated gene. A score of 3 indicated partial completeness, with information drawn from at least one of the sources but lacking key elements. A score of 4 denoted mostly complete responses, with only minor omissions. A score of 5 indicated full completeness, incorporating all relevant variant and gene information from ClinVar, ClinGen, and GenCC, including American College of Medical Genetics and Genomics (ACMG) codes, 21 pathogenicity, and gene- or disease-level details. 2.3.2.2. Automated NLP Evaluation Although human evaluation is considered the gold standard for assessing LLM performance on clinical tasks, 22 , 23 it is costly, labor intensive, and time-consuming (i.e., not scalable). Therefore, we assessed the effectiveness of traditional NLP metrics in evaluating the LLM-generated summaries for genetic variants. We tested several traditional metrics originally developed for machine translation evaluation, including BLEU, 24 which measures n-gram overlap between generated and reference texts; ROUGE, 25 which evaluates recall-oriented overlap of textual units such as n-grams or longest common subsequences; and METEOR, 26 which incorporates synonym matching and stemming to better capture semantic similarity beyond exact word overlap. In addition, we evaluated more recent metrics designed to capture semantic similarity, including cosine similarity of Llama-3-generated sentence embeddings 27 and AlignScore, 28 which assesses factual consistency between generated and reference texts. We calculated the correlation between automated NLP metric scores and human evaluation scores using Spearman’s rank correlation coefficient, which was selected due to its widespread application in previous studies examining the relationship between human assessments and automated NLP metrics. 22 , 29 We tested metrics designed for machine translation (BLEU, 24 ROUGE, 25 and METEOR 26 ), and more recent metrics that measure the semantic similarity (cosine similarity of Llama 3-generated embeddings, 27 AlignScore 28 ). We calculated the correlation between automated NLP metric scores and human evaluation scores using Spearman’s rank correlation coefficient. Spearman correlation was selected because it evaluates the monotonic relationship between two variables based on their rank order rather than assuming a linear relationship or normally distributed scores, making it suitable for comparing ordinal human ratings (e.g., Likert-scale scores) with automated metric outputs. Additionally, Spearman correlation has been widely used in prior studies evaluating the alignment between human assessments and automated NLP metrics for summarization tasks. 18 , 22 , 29 In practice, this approach allows us to assess whether automated metrics rank LLM-generated summaries in a manner consistent with expert judgment, which is critical for determining whether automated metrics can support scalable evaluation of model performance. 2.3.3. Comparison and Evaluation of Precision Grounding and Baseline Following model selection, we augmented the best performing model with Precision Grounding and baseline methods separately. During precision grounding, the CATT-generated .CSV and .TXT files were uploaded as attachments, and the model was informed of their presence via the additional instruction section of the prompt. As baseline approaches, we evaluated MedRAG and web-search grounding. MedRAG was selected as a representative biomedical RAG framework because it integrates external biomedical domain databases and is widely established in medical question-answering tasks. When implementing MedRAG, we utilized the StarPearl and Textbook knowledge bases. Other corpora supported by MedRAG, such as PubMed, were excluded because their massive scale would significantly increase retrieval latency and computational costs for this study. 30 2.4. Error Analysis During error analysis, we categorized errors into two types based on their potential impact on clinical interpretation: 1) Clinically significant errors: these errors could alter the variant classification in a way that may lead to inappropriate clinical decisions. This category includes factual inaccuracies or hallucinations that misrepresent the variant’s pathogenicity when compared to the current knowledge (e.g., from ClinVar). For example, if the LLM incorrectly states that a benign variant is pathogenic, this could result in unnecessary interventions or anxiety for patients. 2) Clinically negligible errors: these are minor inaccuracies or misstatements that do not affect the overall interpretation or classification of the variant. Examples of these errors include numerical errors or suggesting a variant classification code that is inappropriate while maintaining the correct overall pathogenicity. Such errors are unlikely to mislead a clinician or influence medical decision-making, as the core pathogenicity assessment remain aligned with current evidence. 3. RESULTS 3.1. Evaluation of The Effectiveness of Precision Grounding Characteristics of the 50 included genetic variants are illustrated in Supplementary Table S1 , which includes their pathogenicity information, and their information density in databases. 3.1.1. LLM Selection The evaluation result is illustrated in Supplementary Table S2 . GPT-4o demonstrated the highest average accuracy (4.7), followed by DeepSeek R1 (4.1), OpenAI o1 (3.5), and Claude 3.5 Sonnet (3.5). For completeness, DeepSeek R1 scored the highest (4.9), followed by GPT-4o (4), o1 (3.8), and Claude 3.5 Sonnet (3.2). Since GPT-4o achieved the highest accuracy, it was selected to test the effectiveness of CATT. Detailed prompt and responses from LLMs during this process are illustrated in Supplementary Table S3 . Supplementary Table S4 contains notes and scores of each variant from our genetic expert when evaluating the LLM’s accuracy. 3.1.2. Evaluation Method Selection Gold standard genetic summaries written by our genetic expert (AN) are provided in the Supplementary Table S5 , which were used for calculating NLP metrics. We found that all automated NLP metrics exhibited low Spearman correlations with human evaluations ( Figure 2 ). For accuracy, the highest (though weak) positive correlation was observed with ROUGE-2 (ρ = 0.1295, p = 0.3702), followed by AlignScore-base 28 (ρ = 0.1490, p = 0.3018); however, neither correlation was statistically significant, indicating a weak and unreliable association with human-rated accuracy. In contrast, for completeness, several metrics showed statistically significant negative correlations with human ratings: BLEU-2 (ρ = –0.3435, p = 0.0146), BLEU-4 (ρ = –0.3321, p = 0.0185), METEOR (ρ = –0.3134, p = 0.0267), and ROUGE-L (ρ = –0.3087, p = 0.0292). These findings indicate that none of the automated metrics reliably captured human judgements of summary accuracy. Moreover, the significant inverse correlations with completeness suggest that, in the genetic domain, lower BLEU, METEOR, and ROUGE-L scores may correspond to higher human-perceived completeness—possibly reflecting limitations of these metrics in evaluating abstractive, semantically rich summaries. Therefore, due to the weak correlation between NLP metrics with gold standard human evaluation, we decided to use human evaluation. Figure 2. Spearman’s Correlation between Human Evaluation and NLP Metrics Evaluation. Open in a new tab The left side represents the correlation of accuracy between human evaluation and NLP metrics, while the right side is the correlation of completeness between human evaluation and NLP metrics. (A) shows the Spearman correlation coefficients between human ratings (accuracy and completeness) and automated NLP metrics. (B) presents the corresponding p-values indicating statistical significance. Overall, the correlations with human-judged accuracy were weak and non-significant across all metrics (e.g., BLEU-2: ρ = 0.0225, p = 0.8767; AlignScore-base: ρ = 0.1490, p = 0.3018). In contrast, several metrics exhibited statistically significant correlations with completeness, though all were negative: BLEU-2 (ρ = –0.3435, p = 0.0146), BLEU-4 (ρ = –0.3321, p = 0.0185), METEOR (ρ = –0.3134, p = 0.0267), and ROUGE-L (ρ = –0.3087, p = 0.0292). 3.1.3. Comparison and Evaluation of Precision Grounding and Baseline MedRAG did not provide useful variant summary information, but only some information about where to look for the answer (e.g., asking the user to check ClinVar database). Detailed responses produced using MedRAG are included in the Supplementary Table S6 . This is, in part, because MedRAG is implicitly encouraged to avoid unsupported claims and provide answers only if supported by retrieved evidence. Therefore, we only scored and evaluated the other baseline method: web-search grounding. Table 1 shows the performance of the best-performing model (GPT 4o) on 50 variants with precision grounding and web-search grounding. We found precision grounding could largely improve the accuracy from 4.02 to 4.76 and completeness from 4.10 to 4.94. In addition, precision grounding eliminated clinically harmful hallucination such as summarization of the wrong variant or misinterpretation of the pathogenicity of a variant. Detailed prompt and responses from LLMs during this process are illustrated in Supplementary Table S7 . Supplementary Table S8 contains notes and scores of each variant from our genetic expert during the evaluation. Table 1. Manual Evaluation of LLM (GPT-4o) Performance Using Web-Search vs. Knowledge Grounding Approaches (n=50 variants) Measurement GPT-4o + Web Search Grounding GPT-4o + Knowledge Grounding Using CATT Average Accuracy 4.02 4.76 Average Completeness 4.10 4.94 Open in a new tab 3.2. Error Analysis Based on the evaluation of the ten variants during the LLM selection phase, as illustrated in Supplementary Table S9 , we found O1 could misclassify the genetic variant pathogenicity and miss information from ClinVar when summarizing genetic variant. Claude 3.5 Sonnect could classify the pathogenicity incorrectly, report a wrong number of entries, and miss information from ClinVar. DeepSeek R1could also incorrectly classify the pathogenicity and report wrong number of entries. The most accurate model GPT-4o could misclassify the pathogenicity as well. Additionally, GPT 4o’s summarization sometimes misses disease information, information from ClinVar, or minor interpretation information such as ACMG code. In addition to aforementioned errors and hallucinations, during the phase of testing the effectiveness of precision grounding using 50 variants, we found both GPT-4o + web-search grounding and GPT-4o + precision grounding could overstate the pathogenicity. Nevertheless, GPT-4o + web-search grounding sometimes summarized the wrong variant ( Table 2 ). Precision grounding reduced clinically harmful hallucination, including pathogenicity misclassification and summarizing the wrong variant. Additionally, precision grounding made the summary much more completed, which further improved its trustworthiness and clinical utility. Table 2. Error Analysis Measures Error Category Specific Error Error Summary Example * GPT-4o + Web-Search Grounding GPT-4o + Precision Grounding Accuracy Clinically significant errors Incorrect classification of pathogenicity X 360767 Summarized wrong variant X 3020178 Clinically negligible errors Overstated pathogenicity X X 1285628; 50866 Incorrect population frequency X X 370854; 13507 Incorrect codes X X 444210; 370854 Report incorrect number of submissions in databases X 414302 Completeness Missing disease information X 2298 Missing information of entries in a database X X 372641; 5760 Missing ID information (e.g., ACMG code) X 157939 Open in a new tab “X” means that the model made the specific type of error(s). * Detailed LLM response can be found in the row with corresponding Variation ID values in Supplementary Table S6 ; detailed error comments can be found in Supplementary Table S7 . 4. DISCUSSION LLMs have demonstrated remarkable capabilities in natural language understanding and generation; however, they are prone to producing content that appears factual but is ungrounded—a phenomenon known as hallucination. In genomic medicine, such inaccuracies can lead to misinterpretations of genetic variants, potentially resulting in erroneous clinical decisions. To reduce this risk, various techniques have been proposed for enhance LLM outputs by incorporating external information. 31 Nevertheless, commonly-used Web-Search Grounding grabs information from the Internet, which might be low quality and without validation; Vector Database Grounding needs the creation and maintenance of the vector database, but such databases does not exist in genomic domain, and it is costly and labor-intensive to create and maintain such a domain-specific vector database. In this study, we proposed Precision Grounding, which uses a query tool to retrieve evidence from ready-to-use, domain-specific, and well-maintained databases to guide LLM outputs. We developed CATT, an open-source tool that integrates authoritative variant databases and enables users to query and retrieve relevant information for specific variants using Variation IDs. When applied to variant summarization tasks, Precision Grounding significantly improves both accuracy and completeness while reducing clinically harmful hallucinations. This improvement facilitates the clinical utility of precision grounding and LLM, offering an important step toward providing healthcare professionals with more trustworthy and actionable variant interpretations. Precision grounding substantially improves the accuracy and completeness of LLM outputs for genetic variant summarization. Among the evaluated models, GPT-4o achieved the highest accuracy. This is partially attributed to its ability to perform live web searches, unlike O1, Claude 3.5, and DeepSeek R1, which did not have the function of web-search grounding at the time we used them. Notably, O1, Claude 3.5 Sonnet, and DeepSeek R1 occasionally misclassified pathogenicity of a variant, which could lead to harmful consequences in clinical practice, underscoring the need for factual grounding in LLM outputs. Despite GPT-4o’s superior performance, hallucination persisted, including misclassifying pathogenicity and summarizing the wrong variant. However, when we replaced web-based retrieval with CATT-based precision grounding, these hallucinations were eliminated. Regarding completeness, our precision grounding approach provides an LLM with comprehensive domain specific data from curated databases. This enables the LLM to receive critical annotations such as ACMG codes and disease association information, which were frequently missed by using web-search grounding. In addition, our evaluation of MedRAG showed that when relevant variant-specific evidence was not retrieved from its document corpus, the model declined to generate detailed summaries and instead provided generic guidance on where to search for information. This finding further suggests that the usefulness of LLM outputs in this task is largely determined by the quality and relevance of the retrieved evidence. These results demonstrated the effectiveness of our precision grounding strategy and CATT toolkit in improving the reliability and comprehensiveness of LLM-generated summaries in the genetic domain. Consistent with prior studies, 22 , 29 we observed weak correlations between automated natural language processing (NLP) evaluation metrics and expert human assessments, highlighting a critical limitation of current approaches for evaluating LLM-generated text. Traditional metrics such as BLEU, 24 ROUGE, 25 and METEOR, 26 primarily measure surface-level similarity between generated and reference texts and therefore often fail to capture semantically correct paraphrasing, factual accuracy, and clinical relevance, which are criteria that are central to expert assessment in genomic interpretation tasks. Although we additionally evaluated semantic similarity metrics such as LLaMA-3 embedding cosine similarity and AlignScore, these approaches also showed limited alignment with expert ratings. While human evaluation remains the gold standard due to its nuanced and context-aware judgment, it is resource-intensive and difficult to scale. 32 Our findings highlight the need for task-specific and clinically contextual automated evaluation methods designed to better align with expert judgment for particular clinical applications. Developing such metrics will likely require domain expertise to identify and verify key clinical attributes (e.g., pathogenicity classification, gene–disease relationships and supporting evidence) and to design evaluation criteria that reflect expert interpretation workflows. The detailed human evaluation results and LLM-generated summaries released in our supplementary materials may therefore provide a valuable reference dataset for future research aimed at developing and benchmarking automated evaluation methods for genetic variant summarization using LLMs. To enhance clinical applicability, we envision integrating the proposed Precision Grounding framework into existing clinical decision-support systems (CDSS) within genomic medicine. By embedding our CATT tool and grounding strategy into variant interpretation workflows, clinical geneticists and molecular pathologists can generate accurate and complete summaries more efficiently from domain-specific databases, reducing reliance on potentially inconsistent web sources. This aligns with prior efforts that emphasize the importance of explainability and trust in AI-assisted variant interpretation. 5 , 6 Integration could occur at the point of variant review, where LLM-generated summaries serve as a first draft for expert validation, streamlining interpretation and reducing documentation burden, which is a known bottleneck in clinical genomics. 1 In practice, the generated summaries could be presented together with the underlying structured evidence retrieved by CATT, allowing clinicians to quickly review the supporting database information before incorporating the summary into clinical reports. Such human-in-the-loop workflows ensure that expert oversight remains central to the interpretation process while leveraging LLMs to reduce repetitive literature review and documentation effort. 33 This approach supports efficient clinical review while maintaining transparency and accountability in variant interpretation. Additionally, the open-source and modular design of CATT supports interoperability with laboratory information systems (LIS) and EHRs, facilitating broader deployment. Such real-world implementation holds promises for improving efficiency, reducing manual burden, and enhancing the trustworthiness of LLM outputs in clinical genomics. The integration of external knowledge sources into LLM workflows also raises important considerations related to data security and governance. In clinical environments, protecting patient privacy and ensuring compliance with institutional and regulatory policies are critical when deploying AI systems. Specifically, AI deployment must strictly adhere to regulations like HIPAA (in the U.S.) or GDPR (in the EU) to protect sensitive patient data. Beyond legal requirements, institutional policies ensure that AI tools are used ethically, securely, and without compromising patient trust. 34 In our framework, Precision Grounding retrieves information from publicly available genomic databases (e.g., ClinGen, ClinVar, and GenCC) using variant identifiers rather than patientspecific clinical data, which reduces the risk of exposing protected health information during the retrieval process. Additionally, the CATT tool operates as a modular query system that can be deployed within secure institutional computing environments, enabling healthcare organizations to maintain control over data access and governance policies. Future implementations of Precision Grounding in clinical systems should continue to prioritize secure infrastructure, auditability of data sources, and compliance with healthcare data protection standards when integrating LLM technologies into clinical workflows. This study has several limitations. Due to the high cost of expert evaluation, our analysis was limited to a sample of 50 variants. Broader validation of CATT using a larger and more diverse set of variants will be necessary to comprehensively assess its effectiveness. Additionally, the manual evaluation was primarily conducted by a single domain expert using predefined rubric, which may introduce subjective bias despite efforts to standardize the evaluation criteria through expert consultation. To enhance transparency and facilitate independent assessment, all LLM-generated summaries, evaluation scores, and detailed reviewer notes are provided in the supplementary materials , allowing readers to examine the results and interpret the model performance independently. Additionally, we were not able to assess all existing evaluation method exhaustively, and it is possible that some other methods like BLEURT 35 or using LLM as a judge 36 may have a better alignment with human expert evaluation. Additionally, we haven’t exhaustively tested the effectiveness of other existing grounding methods such as Vector Database Grounding, 31 REALM, 37 and ReAct. 38 Although these algorithms have not been integrated into widely-used LLMs as a function like Web-Search Grounding, they also have the potential to mitigate hallucinations in the genetic variant summarization task. Future work should focus on extending Precision Grounding to additional domains by developing domain-specific query tools integrating expert-identified databases tailored to each domain. Because the proposed framework is model-agnostic, future studies should also evaluate its performance across a broader range of LLMs, including both proprietary and open-source systems, to assess cross-model robustness, reduce potential vendor-specific bias, and mitigate reproducibility issues of proprietary models. Future work will also explore systematic prompt perturbation analyses to further evaluate the robustness of Precision Grounding under different prompting variations. To improve usability and scalability, we plan to develop a fully automated pipeline that eliminates manual steps—such as extracting files from CATT and manually attaching them to LLMs via web interfaces— and enhances the software’s reproducibility and FAIRness (Findability, Accessibility, Interoperability, and Reusability). 39 , 40 We also aim to implement a graphical user interface to improve CATT’s accessibility for nontechnical users. The development of an automated, NLP-based evaluation framework is essential for scalable assessment of genetic variant summarization. Such a system would enable large-scale evaluation of precision grounding in the genomic domain. Additionally, methods to automatically detect hallucinations in LLMgenerated variant summaries are warranted. 5. CONCLUSION Accurate genetic variant summarization is essential for timely and reliable clinical decision-making. While LLMs hold substantial promises for automating this task, their susceptibility to hallucinations presents a major barrier to clinical adoption. To address this, we proposed Precision Grounding and developed CATT, an open-source tool that retrieves variant-specific information from curated, evidence-based genomic databases. Our findings demonstrate that the CATT-assisted Precision Grounding framework significantly improves the accuracy and completeness of LLM-generated summaries and reduces clinically harmful hallucinations compared to existing factual grounding approaches such as web-search grounding. Importantly, this approach supports real-world implementation by enabling integration into clinical decision-support systems and laboratory workflows, where variant interpretation must be both rapid and reliable. Its modular, interoperable architecture facilitates adoption across institutions and platforms. Moreover, the weak correlations observed between standard NLP metrics and expert ratings highlight the need for improved automated evaluation methods aligned with clinical utility. All intermediate results and evaluation materials are provided to ensure transparency and reproducibility. Although our current evaluation focuses on the genomic domain, the Precision Grounding framework is readily extensible to other high-stakes clinical areas where trustworthy, grounded LLM outputs are essential. Supplementary Material 1 NIHMS2163911-supplement-1.docx (343.3KB, docx) 2 NIHMS2163911-supplement-2.xlsx (54.3KB, xlsx) 3 NIHMS2163911-supplement-3.xlsx (158.2KB, xlsx) Highlights. Ungrounded LLMs may generate unreliable genetic variant summaries. We present a methodology to mitigate LLM hallucinations in variant summarization. Our proposed methodology improved accuracy and completeness in LLM’s output. Errors commonly made by LLMs and mitigated by our method were highlighted. Summary Table. What was already known on the topic: Large language models (LLMs) have demonstrated strong capabilities in text summarization and reasoning but are prone to factual hallucinations that can limit their clinical reliability. Web-Search Grounding has been widely used to reduce hallucinations, but retrieved information from the internet often lacks quality control and may be clinically misleading. In genomic medicine, hallucinated or incorrect variant interpretations can lead to clinically harmful outcomes, highlighting the need for validated, evidence-based grounding strategies. What this study added to our knowledge: We developed an open-source tool CATT to integrate curated genomic databases, and proposed Precision Grounding, which augments LLMs with CATT-generated evidence-based factual information. Precision grounding significantly improved accuracy and completeness compared with Web-Search Grounding. Precision Grounding reduced clinically harmful hallucinations, such as misclassified pathogenicity. Acknowledgement Research was supported by NIH-NHGRI (U24HG006834–10S1), NIH-NIA (R01AG080429), NIH-NLM (R01LM014239), and OpenAI Researcher Access Program. Footnotes Declaration of interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. References 1. Harrison SM, Biesecker LG, Rehm HL. Overview of Specifications to the ACMG/AMP Variant Interpretation Guidelines. Curr Protoc Hum Genet. 2019;103(1):e93. doi: 10.1002/cphg.93 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 2. Green RC, Shah N, Genetti CA, et al. Actionability of unanticipated monogenic disease risks in newborn genomic screening: Findings from the BabySeq Project. Am J Hum Genet. 2023;110(7):1034–1045. doi: 10.1016/j.ajhg.2023.05.007 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 3. Machini K, Ceyhan-Birsoy O, Azzariti DR, et al. Analyzing and Reanalyzing the Genome: Findings from the MedSeq Project. Am J Hum Genet. 2019;105(1):177–188. doi: 10.1016/j.ajhg.2019.05.017 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 4. Diao JA, Kohane IS, Manrai AK. Biomedical informatics and machine learning for clinical genomics. Hum Mol Genet. 2018;27(R1):R29–R34. doi: 10.1093/hmg/ddy088 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 5. De La Vega FM, Chowdhury S, Moore B, et al. Artificial intelligence enables comprehensive genome interpretation and nomination of candidate diagnoses for rare genetic diseases. Genome Med. 2021;13(1):153. doi: 10.1186/s13073-021-00965-0 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 6. McInnes G, Sharo AG, Koleske ML, et al. Opportunities and challenges for the computational interpretation of rare variation in clinically important genes. Am J Hum Genet. 2021;108(4):535–548. doi: 10.1016/j.ajhg.2021.03.003 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 7. Farquhar S, Kossen J, Kuhn L, Gal Y. Detecting hallucinations in large language models using semantic entropy. Nature. 2024;630(8017):625–630. doi: 10.1038/s41586-024-07421-0 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 8. Du X, Novoa-Laurentiev J, Plasek JM, et al. Enhancing early detection of cognitive decline in the elderly: a comparative study utilizing large language models in clinical notes. EBioMedicine. 2024;109:105401. doi: 10.1016/j.ebiom.2024.105401 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 9. Zhang G, Xu Z, Jin Q, et al. Leveraging long context in retrieval augmented language models for medical question answering. NPJ Digit Med. 2025;8:239. doi: 10.1038/s41746-025-01651-w [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 10. Masanneck L, Meuth SG, Pawlitzki M. Evaluating base and retrieval augmented LLMs with document or online support for evidence based neurology. npj Digit Med. 2025;8(1):1–5. doi: 10.1038/s41746-025-01536-y [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 11. Wang D, Zhang S. Large language models in medical and healthcare fields: applications, advances, and challenges. Artif Intell Rev. 2024;57(11):299. doi: 10.1007/s10462-024-10921-0 [ DOI ] [ Google Scholar ] 12. Li X, Zhao R, Chia YK, et al. Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources. arXiv. Preprint posted online February 21, 2024:arXiv:2305.13269. doi: 10.48550/arXiv.2305.13269 [ DOI ] [ Google Scholar ] 13. Kenthapadi K, Sameki M, Taly A. Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey). In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. KDD ‘24. Association for Computing Machinery; 2024:6523–6533. doi: 10.1145/3637528.3671467 [ DOI ] [ Google Scholar ] 14. Abo El-Enen M, Saad S, Nazmy T. A survey on retrieval-augmentation generation (RAG) models for healthcare applications. Neural Comput & Applic. 2025;37(33):28191–28267. doi: 10.1007/s00521-025-11666-9 [ DOI ] [ Google Scholar ] 15. Yang W, Some L, Bain M, Kang B. A comprehensive survey on integrating large language models with knowledge-based methods. Knowledge-Based Systems. 2025;318:113503. doi: 10.1016/j.knosys.2025.113503 [ DOI ] [ Google Scholar ] 16. Fui-Hoon Nah F, Zheng R, Cai J, Siau K, Chen L. Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration. Journal of Information Technology Case and Application Research. 2023;25(3):277–304. doi: 10.1080/15228053.2023.2233814 [ DOI ] [ Google Scholar ] 17. Du X, Anna N, Michael O, Aronson S, Lebo M, Li Z. CATT - ClinGen AI data Transformation Tool. Published online March 2026. doi: 10.5281/zenodo.18896081 [ DOI ] [ Google Scholar ] 18. Du X, Zhou Z, Wang Y, et al. Testing and evaluation of generative large language models in electronic health record applications: a systematic review. J Am Med Inform Assoc. Published online January 13, 2026:ocaf233. doi: 10.1093/jamia/ocaf233 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 19. Bedi S, Liu Y, Orr-Ewing L, et al. Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review. JAMA. 2025;333(4):319–328. doi: 10.1001/jama.2024.21700 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 20. Albaum G The Likert Scale Revisited. Market Research Society Journal. 1997;39(2):1–21. doi: 10.1177/147078539703900202 [ DOI ] [ Google Scholar ] 21. Richards S, Aziz N, Bale S, et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med. 2015;17(5):405–424. doi: 10.1038/gim.2015.30 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 22. Van Veen D, Van Uden C, Blankemeier L, et al. Adapted large language models can outperform medical experts in clinical text summarization. Nat Med. 2024;30(4):1134–1142. doi: 10.1038/s41591-024-02855-5 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 23. Tam TYC, Sivarajkumar S, Kapoor S, et al. A framework for human evaluation of large language models in healthcare derived from literature review. npj Digit Med. 2024;7(1):1–20. doi: 10.1038/s41746-024-01258-7 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 24. Papineni K, Roukos S, Ward T, Zhu WJ. Bleu: a Method for Automatic Evaluation of Machine Translation. In: Isabelle P, Charniak E, Lin D, eds. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics; 2002:311–318. doi: 10.3115/1073083.1073135 [ DOI ] [ Google Scholar ] 25. Lin CY. ROUGE: A Package for Automatic Evaluation of Summaries. In: Text Summarization Branches Out. Association for Computational Linguistics; 2004:74–81. Accessed March 2, 2025. https://aclanthology.org/W04-1013/ [ Google Scholar ] 26. Banerjee S, Lavie A. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In: Goldstein J, Lavie A, Lin CY, Voss C, eds. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. Association for Computational Linguistics; 2005:65–72. Accessed March 2, 2025. https://aclanthology.org/W05-0909/ [ Google Scholar ] 27. Aynetdinov A, Akbik A. SemScore: Automated Evaluation of Instruction-Tuned LLMs based on Semantic Textual Similarity. arXiv. Preprint posted online February 5, 2024:arXiv:2401.17072. doi: 10.48550/arXiv.2401.17072 [ DOI ] [ Google Scholar ] 28. Zha Y, Yang Y, Li R, Hu Z. AlignScore: Evaluating Factual Consistency with a Unified Alignment Function. arXiv. Preprint posted online May 26, 2023:arXiv:2305.16739. doi: 10.48550/arXiv.2305.16739 [ DOI ] [ Google Scholar ] 29. Tie X, Shin M, Pirasteh A, et al. Personalized Impression Generation for PET Reports Using Large Language Models. J Digit Imaging Inform med. 2024;37(2):471–488. doi: 10.1007/s10278-024-00985-3 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 30. Xiong G, Jin Q, Lu Z, Zhang A. Benchmarking Retrieval-Augmented Generation for Medicine. In: Ku LW, Martins A, Srikumar V, eds. Findings of the Association for Computational Linguistics ACL 2024. Association for Computational Linguistics; 2024:6233–6251. https://aclanthology.org/2024.findings-acl.372 [ Google Scholar ] 31. Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ‘20. Curran Associates Inc.; 2020:9459–9474. [ Google Scholar ] 32. Croxford E, Gao Y, Pellegrino N, et al. Current and future state of evaluation of large language models for medical summarization tasks. npj Health Syst. 2025;2(1):1–13. doi: 10.1038/s44401-024-00011-2 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 33. Wang Z, Wang H, Danek B, et al. A perspective for adapting generalist AI to specialized medical AI applications and their challenges. NPJ Digit Med. 2025;8:429. doi: 10.1038/s41746-025-01789-7 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 34. Pantanowitz L, Hanna M, Pantanowitz J, et al. Regulatory Aspects of Artificial Intelligence and Machine Learning. Modern Pathology. 2024;37(12):100609. doi: 10.1016/j.modpat.2024.100609 [ DOI ] [ PubMed ] [ Google Scholar ] 35. Sellam T, Das D, Parikh A. BLEURT: Learning Robust Metrics for Text Generation. In: Jurafsky D, Chai J, Schluter N, Tetreault J, eds. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics; 2020:7881–7892. doi: 10.18653/v1/2020.acl-main.704 [ DOI ] [ Google Scholar ] 36. Gu J, Jiang X, Shi Z, et al. A Survey on LLM-as-a-Judge. arXiv. Preprint posted online March 9, 2025:arXiv:2411.15594. doi: 10.48550/arXiv.2411.15594 [ DOI ] [ Google Scholar ] 37. Guu K, Lee K, Tung Z, Pasupat P, Chang MW. REALM: Retrieval-Augmented Language Model PreTraining. arXiv. Preprint posted online February 10, 2020:arXiv:2002.08909. doi: 10.48550/arXiv.2002.08909 [ DOI ] [ Google Scholar ] 38. Yao S, Zhao J, Yu D, et al. ReAct: Synergizing Reasoning and Acting in Language Models. In: 2022. Accessed July 21, 2025. https://openreview.net/forum?id=WE_vluYUL-X [ Google Scholar ] 39. Du X, Dobrowolski A, Brochhausen M, Garrett TJ, Hogan WR, Lemas DJ. Nextflow4MS-DIAL: A Reproducible Nextflow-Based Workflow for Liquid Chromatography-Mass Spectrometry Metabolomics Data Processing. J Am Soc Mass Spectrom. Published online January 5, 2025. doi: 10.1021/jasms.4c00364 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 40. Du X, Dastmalchi F, Ye H, et al. Evaluating LC-HRMS metabolomics data processing software using FAIR principles for research software. Metabolomics. 2023;19(2):11. doi: 10.1007/s11306-023-01974-3 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials 1 NIHMS2163911-supplement-1.docx (343.3KB, docx) 2 NIHMS2163911-supplement-2.xlsx (54.3KB, xlsx) 3 NIHMS2163911-supplement-3.xlsx (158.2KB, xlsx) ACTIONS View on publisher site PDF (807.6 KB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• Augmenting large language models with chemistry tools#16635
• Measuring the Meaning of Genomic Results: Harmonization of the Metric for Case-Level Results in the CSER2 Consortium#321465
• DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models#120472
• Deterministic retrieval recovers biomedical associations lost by language models#234301
• Efficient Guided Generation for Large Language Models#305584