Linking Symptom Inventories Using Semantic Textual Similarity.

Linking Symptom Inventories Using Semantic Textual Similarity - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice J Neurotrauma . 2025 Jun 4;42(11-12):1008–1020. doi: 10.1089/neu.2024.0301 Linking Symptom Inventories Using Semantic Textual Similarity Eamonn Kennedy 1, 2, 3 , Shashank Vadlamani 1, 3 , Hannah M Lindsey 1, 2 , Kelly S Peterson 3, 4 , Kristen Dams O’Connor 5, 6 , Ronak Agarwal 1 , Houshang H Amiri 7, 8 , Raeda K Andersen 9, 10 , Talin Babikian 11, 12 , David A Baron 13 , Erin D Bigler 1, 14, 15 , Karen Caeyenberghs 16 , Lisa Delano-Wood 17, 18 , Seth G Disner 19, 20 , Ekaterina Dobryakova 21, 22 , Blessen C Eapen 23, 24 , Rachel M Edelstein 25 , Carrie Esopenko 5 , Helen M Genova 26 , Elbert Geuze 27 , Naomi J Goodrich-Hunsaker 1 , Jordan Grafman 28, 29 , Asta K Håberg 30, 31 , Cooper B Hodges 32 , Kristen R Hoskinson 33, 34 , Elizabeth S Hovenden 1 , Andrei Irimia 35, 36, 37 , Neda Jahanshad 38 , Ruchira M Jha 39 , Finian Keleher 1 , Kimbra Kenney 40, 41 , Inga K Koerte 42, 43 , Spencer W Liebel 1 , Abigail Livny 44, 45 , Marianne Løvstad 46, 47 , Sarah L Martindale 48, 49 , Jeffrey E Max 50 , Andrew R Mayer 51 , Timothy B Meier 52 , Deleene S Menefee 53, 54 , Abdalla Z Mohamed 55 , Stefania Mondello 56 , Martin M Monti 57, 58 , Rajendra A Morey 59, 60 , Virginia Newcombe 61 , Mary R Newsome 1, 2 , Alexander Olsen 62, 63, 64 , Nicholas J Pastorek 53, 65 , Mary Jo Pugh 66, 67 , Adeel Razi 68, 69, 70 , Jacob E Resch 71 , Jared A Rowland 48, 72, 73 , Kelly Russell 74, 75 , Nicholas P Ryan 16, 76 , Randall S Scheibel 53, 65 , Adam T Schmidt 77, 78 , Gershon Spitz 79, 80 , Jaclyn A Stephens 81, 82 , Assaf Tal 83 , Leah D Talbert 32 , Maria Carmela Tartaglia 84, 85 , Brian A Taylor 86 , Sophia I Thomopoulos 38 , Maya Troyanskaya 53, 65 , Eve M Valera 87, 88 , Harm Jan van der Horn 51 , John D Van Horn 89 , Ragini Verma 90, 91 , Benjamin SC Wade 92 , Willian C Walker 93, 94 , Ashley L Ware 95, 96 , J Kent Werner Jr 40 , Keith Owen Yeates 97 , Ross D Zafonte 98, 99 , Michael M Zeineh 100 , Brandon Zielinski 1, 101, 102 , Paul M Thompson 38, 103 , Frank G Hillary 104, 105 , David F Tate 1, 2 , Elisabeth A Wilde 1, 2 , Emily L Dennis 1, 2, * Author information Article notes Copyright and License information 1 Department of Neurology, University of Utah School of Medicine, Salt Lake City, Utah, USA. 2 George E. Wahlen Veterans Affairs Medical Center, Salt Lake City, Utah, USA. 3 Division of Epidemiology, University of Utah, Salt Lake City, Utah, USA. 4 Veterans Health Administration (VHA) Office of Analytics and Performance Integration (VHA), Washington, District of Columbia, USA. 5 Department of Rehabilitation and Human Performance, Icahn School of Medicine at Mount Sinai, New York, New York, USA. 6 Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, New York, USA. 7 Neuroscience Research Center, Institute of Neuropharmacology, Kerman University of Medical Sciences, Kerman, Iran. 8 Department of Radiology and Nuclear Medicine, Vrije Universiteit Amsterdam, Amsterdam UMC Location VUmc, Amsterdam, The Netherlands. 9 Crawford Research Institute, Shepherd Center, Atlanta, Georgia, USA. 10 Department of Sociology, Georgia State University, Atlanta, Georgia, USA. 11 Department of Psychiatry and Biobehavioral Sciences, Semel Institute for Neuroscience and Human Behavior, UCLA, Los Angeles, California, USA. 12 UCLA BrainSPORT Program, Los Angeles, California, USA. 13 Department of Psychiatry, Center for Behavioral Health and Sport, Western University of Health Sciences, Lebanon, Pomona, California, USA. 14 Department of Psychology and Neuroscience Center, Brigham Young University, Provo, Utah, USA. 15 Department of Psychiatry, University of Utah, Salt Lake City, Utah, USA. 16 Cognitive Neuroscience Unit, School of Psychology, Deakin University, Geelong, Australia. 17 VA San Diego Healthcare System; Center of Stress and Mental Health, and Department of Psychiatry, UC San Diego School of Medicine, San Diego, California, USA. 18 Department of Psychiatry, UC San Diego School of Medicine, San Diego, California, USA. 19 Minneapolis VA Health Care System, Minneapolis, Minnesota, USA. 20 Department of Psychiatry, University of Minnesota, Minneapolis, Minnesota, USA. 21 Center for Traumatic Brain Injury, Kessler Foundation, East Hanover, New Jersey, USA. 22 Rutgers New Jersey Medical School, Newark, New Jersey, USA. 23 VA Greater Los Angeles Health Care System, Los Angeles, California, USA. 24 Division of Physical Medicine and Rehabilitation, David Geffen School of Medicine at UCLA, Los Angeles, California, USA. 25 Department of Psychology, University of Virginia, Charlottesville, Virginia, USA. 26 Center for Autism Research, Kessler Foundation, East Hanover, New Jersey, USA. 27 Ministry of Defence, Brain Research and Innovation Centre, Utrecht, The Netherlands. 28 Shirley Ryan Ability Lab, Chicago, Illinois, USA. 29 Northwestern University, Chicago, Illinois, USA. 30 Faculty of Medicine and Health Sciences, Department of Neuromedicine and Movement Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway. 31 Department for Physical Health and Aging, Norwegian Institute of Public Health, Oslo, Norway. 32 Department of Psychology, Brigham Young University, Provo, Utah, USA. 33 Center for Biobehavioral Health, Abigail Wexner Research Institute at Nationwide Children’s Hospital, Columbus, Ohio, USA. 34 Department of Pediatrics, The Ohio State University College of Medicine, Columbus, Ohio, USA. 35 Ethel Percy Andrus Gerontology Center, Leonard Davis School of Gerontology, University of Southern California, Los Angeles, California, USA. 36 Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, California, USA. 37 Department of Quantitative and Computational Biology, Dornsife College of Arts and Sciences, University of Southern California, Los Angeles, California, USA. 38 Imaging Genetics Center, Stevens Neuroimaging & Informatics Institute, Keck School of Medicine of USC, Marina del Rey, California, USA. 39 Departments of Neurology, Translational Neuroscience, Barrow Neurological Institute, Phoenix, Arizona, USA. 40 Department of Neurology, Uniformed Services University of the Health Sciences, Bethesda, Maryland, USA. 41 National Intrepid Center of Excellence, Walter Reed National Military Medical Center, Bethesda, Maryland, USA. 42 Psychiatry Neuroimaging Laboratory, Brigham & Women’s Hospital, Boston, Massachusetts, USA. 43 Department of Child and Adolescent Psychiatry, University Hospital, Ludwig-Maximilians-Universität, Munich, Germany. 44 Division of Diagnostic Imaging, Sheba Medical Center, Tel-Aviv, Israel. 45 Faculty of Medicine, Tel-Aviv University, Tel-Aviv, Israel. 46 Sunnaas Rehabilitation Centre, Nesodden, Norway. 47 University of Oslo, Oslo, Norway. 48 W. G. (Bill) Hefner VA Healthcare System, Salisbury, North Carolina, USA. 49 Department of Physiology & Pharmacology, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA. 50 Department of Psychiatry, UC San Diego, La Jolla, California, USA. 51 The Mind Research Network, Albuquerque, New Mexico, USA. 52 Department of Neurosurgery, Medical College of Wisconsin, Milwaukee, Wisconsin, USA. 53 Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA. 54 The Menning Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, Texas, USA. 55 Thompson Institute, University of the Sunshine Coast, Birtinya, Australia. 56 Department of Biomedical and Dental Sciences and Morphofunctional Imaging, University of Messina, Messina, Italy. 57 Department of Psychology, University of California Los Angeles, Los Angeles, California, USA. 58 Department of Neurosurgery, Brain Injury Research Center (BIRC), University of California Los Angeles, Los Angeles, California, USA. 59 Brain Imaging and Analysis Center, Duke University, Durham, North Carolina, USA. 60 Department of Psychiatry and Behavioral Sciences, Duke University, Durham, North Carolina, USA. 61 Department of Medicine, University of Cambridge, Cambridge, UK. 62 Department of Psychology, NTNU, Trondheim, Norway. 63 Clinic of Rehabilitation, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway. 64 NorHEAD - Norwegian Centre for Headache Research, NTNU, Trondheim, Norway. 65 H. Ben Taub Department of Physical Medicine and Rehabilitation, Baylor College of Medicine, Houston, Texas, USA. 66 Information Decision-Enhancement and Analytic Sciences Center, VA Salt Lake City, Salt Lake City, Utah, USA. 67 Department of Medicine, University of Utah School of Medicine, Salt Lake City, Utah, USA. 68 Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia. 69 Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom. 70 CIFAR Azrieli Global Scholars Program, CIFAR, Toronto, Canada. 71 Department of Kinesiology, University of Virginia, Charlottesville, Virginia, USA. 72 Department of Neurobiology and Anatomy, Wake Forest School of Medicine, Winston-Salem, North Carolina, USA. 73 Mid-Atlantic Mental Illness Research and Education Center, Durham, North Carolina, USA. 74 Department of Pediatrics and Child Health, University of Manitoba, Winnipeg, Canada. 75 Children’s Hospital Research Institute of Manitoba, Winnipeg, Canada. 76 Brain and Mind Research, Murdoch Children’s Research Institute, Parkville, Australia. 77 Department of Psychological Sciences, Texas Tech University, Lubbock, Texas, USA. 78 Center of Excellence For Translational Neuroscience and Therapeutics, Lubbock, Texas, USA. 79 Monash-Epworth Rehabilitation Research Centre, School of Psychological Sciences, Monash University, Melbourne, Australia. 80 Department of Neuroscience, Central Clinical School, Monash University, Melbourne, Australia. 81 College of Health and Human Sciences, Colorado State University, Fort Collins, Colorado, USA. 82 Molecular Cellular Integrative Neuroscience Program, Colorado State University, Fort Collins, Colorado, USA. 83 Biomedical Engineering, Tel Aviv University, Tel Aviv Israel. 84 University Health Network, Toronto, Canada. 85 Centre for Research in Neurodegenerative Diseases, University of Toronto, Toronto, Canada. 86 Department of Imaging Physics, The University of Texas M. D. Anderson Cancer Center, Houston, Texas, USA. 87 Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, USA. 88 Massachusetts General Hospital, Boston, Massachusetts, USA. 89 Department of Psychology and School of Data Science, University of Virginia, Charlottesville, Virginia, USA. 90 Department of Radiology, University of Pennsylvania, Pennsylvania, USA. 91 Cohen Veterans Bioscience, New York City, New York, USA. 92 Division of Neuropsychiatry and Neuromodulation, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, USA. 93 Department of Physical Medicine & Rehabilitation, Virginia Commonwealth University, Richmond, Virginia, USA. 94 Richmond Veterans Affairs (VA) Medical Center, Central Virginia VA Health Care System, Richmond, Virginia, USA. 95 Department of Psychology, Georgia State University, Atlanta, Georgia, USA. 96 Department of Neurology, University of Utah, Salt Lake City, Utah, USA. 97 Department of Psychology, Alberta Children’s Hospital Research Institute, and the Hotchkiss Brain Institute, University of Calgary, Calgary, Canada. 98 Spaulding Rehabilitation Hospital, Boston, Massachusetts, USA. 99 Department of Physical Medicine & Rehabilitation, Harvard Medical School, Boston, Massachusetts, USA. 100 Department of Radiology, Stanford University, California, USA. 101 Departments of Pediatrics, Neurology, and Neuroscience, University of Florida, Gainesville, Florida, USA. 102 Departments of Pediatrics, and Radiology, University of Utah, Salt Lake City, Utah, USA. 103 Departments of Neurology, Pediatrics, Psychiatry, Radiology, Engineering, and Ophthalmology, USC, Los Angeles, California, USA. 104 Department of Psychology, Penn State University, State College, Pennsylvania, USA. 105 Department of Neurology, Hershey Medical Center, State College, Pennsylvania, USA. * Address correspondence to: Emily L. Dennis, PhD, Department of Neurology, University of Utah School of Medicine, 383 Colorow Drive, Salt Lake City, UT 84108, USA emily.dennis@hsc.utah.edu i ORCID ID ( https://orcid.org/0000-0003-3826-854X ). ii ORCID ID ( https://orcid.org/0000-0001-7009-6843 ). iii ORCID ID ( https://orcid.org/0000-0002-6005-2115 ). iv ORCID ID ( https://orcid.org/0000-0003-4044-9753 ). v ORCID ID ( https://orcid.org/0000-0002-9254-9388 ). vi ORCID ID ( https://orcid.org/0000-0003-1281-9286 ). vii ORCID ID ( https://orcid.org/0000-0002-7036-4198 ). viii ORCID ID ( https://orcid.org/0000-0003-2396-5609 ). ix ORCID ID ( https://orcid.org/0000-0002-4495-2754 ). x ORCID ID ( https://orcid.org/0000-0001-6044-9035 ). xi ORCID ID ( https://orcid.org/0000-0001-8691-3860 ). xii ORCID ID ( https://orcid.org/0000-0003-4196-7763 ). xiii ORCID ID ( https://orcid.org/0000-0002-3655-3411 ). xiv ORCID ID ( https://orcid.org/0000-0002-0878-8889 ). xv ORCID ID ( https://orcid.org/0000-0001-5813-3939 ). xvi ORCID ID ( https://orcid.org/0000-0002-8390-4915 ). xvii ORCID ID ( https://orcid.org/0000-0002-6940-9096 ). xviii ORCID ID ( https://orcid.org/0000-0003-0213-1920 ). xix ORCID ID ( https://orcid.org/0000-0001-7112-4009 ). This paper was previously submitted to a preprint server: https://arxiv.org/abs/2309.04607 . Collection date 2025 Jun. Copyright 2025, Mary Ann Liebert, Inc., publishers PMC Copyright notice PMCID: PMC12169902  PMID: 40200899 Abstract An extensive library of symptom inventories has been developed over time to measure clinical symptoms of traumatic brain injury (TBI), but this variety has led to several long-standing issues. Most notably, results drawn from different settings and studies are not comparable. This creates a fundamental problem in TBI diagnostics and outcome prediction, namely that it is not possible to equate results drawn from distinct tools and symptom inventories. Here, we present an approach using semantic textual similarity (STS) to link symptoms and scores across previously incongruous symptom inventories by ranking item text similarities according to their conceptual likeness. We tested the ability of four pretrained deep learning models to screen thousands of symptom description pairs for related content—a challenging task typically requiring expert panels. Models were tasked to predict symptom severity across four different inventories for 6,607 participants drawn from 16 international data sources. The STS approach achieved 74.8% accuracy across five tasks, outperforming other models tested. Correlation and factor analysis found the properties of the scales were broadly preserved under conversion. This work suggests that incorporating contextual, semantic information can assist expert decision-making processes, yielding broad gains for the harmonization of TBI assessment. Keywords: artificial intelligence, harmonization, traumatic brain injury, semantic textual similarity, symptom inventories Introduction Self-reported symptom inventories are essential tools across clinical and research settings. 1–6 Standard clinical practice for patients with mild traumatic brain injury (TBI) enacts a symptom-based approach to direct treatment, but documenting heterogenous symptoms is a complex process. 7 , 8 As a result, a wide variety of self-reported symptom inventories has been developed over time, each with distinct items, phrasings, use cases, and reference periods. 2–6 To give one example, a recent report found at least nine different symptom inventories are used by athletic trainers to assess sports-related concussion. 9 Beyond TBI, hundreds of distinct symptom assessments are in common use, which undermines reproducibility and our capacity to synthesize findings drawn from distinct sources. 10 To alleviate this problem, clinicians who regularly use these instruments form expert panels to identify similar symptomatology across different instruments and cocalibrate results. 11–14 This expert panel-based approach is labor-intensive, and its subjectivity can introduce noise and bias. 15 Despite expert panel efforts to identify standard definitions and controlled terminologies, only modest reductions in the variety of instruments have been achieved. 16 , 17 Meanwhile, the number of comparisons needed is large, 18 while efforts to harmonize inventories are slow, costly, and can involve hundreds of experts. 19 Supplementing human expertise with artificial intelligence (AI) tools has the potential to enhance diagnosis, reduce disease-related burden, and automate laborious processes. 20–23 Beyond diagnosis, deep learning has the potential to assist with complex clinical and research processes where intuition and domain expertise are required. 24 These advances have renewed interest in various applications of transformer models to process clinical text by fine-tuning contextual embedding models on text semantics corpora, incorporating biomedical metadata using transformer embeddings, and testing various embeddings schema on biomedical source vocabularies. 25–27 Here, we report an international collaboration between neuroscientists, clinicians, and AI experts to robustly link symptom inventories using semantic textual similarity (STS). Instead of relying on predictive models to estimate scores, our approach leverages the meaningful relationship between descriptions of symptoms to identify related content. 28 We demonstrate how pretrained natural language processing (NLP) models can offer a rapid and accurate means to quantify the relationships between thousands of symptoms across measures. We focus on four inventories, two that are commonly used to assess both TBI and general symptoms: The Brief Symptom Inventory-18 (BSI-18) and The Symptom Checklist-90-Revised (SCL-90-R), as well as two inventories specific to brain injury: The Neurobehavioral Symptom Inventory (NSI) and The Rivermead Post-Concussion Symptoms Questionnaire (RPQ). 2–6 We prioritized TBI because it is frequently clinically assessed using self-reported symptom inventories. 7 In addition, TBI research lacks the consensus within the field that research into other conditions have achieved, perhaps due to the heterogeneity of TBI clinical presentation and the multitude of existing TBI instruments 29 . We hypothesized that an approach incorporating contextual semantic information would outperform traditional and machine learning models when tasked to predict participant scores on symptoms across different inventories. This hypothesis was based on our observation that the application of clinical intuition can sometimes better predict and explain trends in noisy medical data than pure learning models. Following AI safety and reporting requirements, 30 , 31 we tested this hypothesis by assessing the ability of semantically linked items to estimate cross-inventory symptom scores for thousands of patients with TBI who were dually assessed on different inventories. The resulting analysis pipeline is available as a free online tool ( https://enigma-tools.shinyapps.io/symptom-inventories-calculator/ ) that can convert scores across previously incompatible symptom inventories. However, this tool is designed for research and testing purposes and is not currently tested for clinical use. This study presents a tangible example of how deep learning tools may help to mitigate long-standing health data compatibility issues in TBI research and practice as well as for general health assessments. Methods Inventory data sources This secondary mega-analysis 32 study petitioned collaborators for item-level data, drawing from the Enhancing NeuroImaging Genetics Through Meta-Analysis Brain Injury working group 33 , 34 and the Long-term Impact of Military-relevant Brain Injury Consortium—Chronic Effects of Neurotrauma Consortium. 35 We also included public data from the Federal Interagency Traumatic Brain Injury Research Informatics System (FITBIR). 36 We obtained 16 datasets that included different combinations of symptom inventories (see Supplementary Table S1 ). Multiple examples of inventory items are provided in the results. Data quality and consistency were confirmed during discussions among authors who collected the primary data. The University of Utah provided overall institutional review board (IRB) approvals and data use agreements. All self-reported measures were completed or administered in English. Measures Comprehensive details of the four self-reported symptom inventories are provided in Supplementary Data S1 . Briefly, we assessed two TBI-related inventories, the NSI 4 and the RPQ, 2 , 3 and two general symptom inventories commonly used for TBI: The 18-item BSI-18 5 and The SCL-90-R. 6 The 22-item NSI 4 evaluates cognitive, somatic, and emotional symptoms commonly experienced by adults following a brain injury, including headache, dizziness, irritability, and difficulty concentrating. Symptom frequency and severity are measured on a 5-point Likert-Scale, where the respondent indicates the degree to which they were disturbed by each symptom over the past two weeks from 0 ( none ) to 4 ( very severe ). The 16-item RPQ 2 , 3 measures the presence and severity of commonly reported TBI symptoms, across somatic, cognitive, and emotional domains, including headache, dizziness, fatigue, irritability, and concentration difficulties. Using a 5-point Likert scale from 0 ( not experienced ) to 4 ( severe problem ), respondents are instructed to rate the severity of each symptom experienced within the past 24 h, relative to their experience of the symptom before injury. The 18-item BSI-18 5 is a shortened version of the SCL-90-R and the original 53-item BSI. The BSI-18 is designed to efficiently and broadly assess psychological symptoms in both healthy and patient populations. The BSI-18 consists of items rated on a 5-point scale, ranging from 0 ( not at all ) to 4 ( extremely ), indicating how much the problems distressed or bothered respondents over the past 7 days. 5 The SCL-90-R 6 is a widely used self-report measure designed to assess the presence, severity, and frequency of 90 broad psychological symptoms and measures of emotional distress and, like the BSI-18, is not just specific to TBI. Respondents rate each item on a 5-point scale from 0 ( not at all) to 4 ( extremely ), indicating how much they have been bothered or distressed by the symptom over the past week. Semantic textual similarity This study used the similarity of question-level text descriptions to identify related items across inventories. This is a challenging task because symptom descriptions can have similar meanings, yet share no words in common. For example, “Vision problems,” “blurring,” “trouble seeing,” and “Light sensitivity” all relate to vision/ocular symptoms, but they do not include the same words. Therefore, a traditional bag-of-words model that merely counts terms would not necessarily calculate these symptoms text as related (see Supplementary Data S2 ). Advancements in NLP have yielded tools that can rapidly score the semantic similarity of text, such as transformer models trained on a large corpus of text to encode and represent text strings within an embedded feature space. 37–40 These approaches can represent text descriptions as feature vectors via pretrained embeddings which offers two main advantages: (1) Sentences of arbitrary length are converted to an embedded feature vector of prespecified length. This means different lengths of text can be directly compared as representations of prespecified length; and (2) sentences closer in the embedding space are more semantically similar, so the distance between feature vectors measures the meaningful similarity of text. AI safety and reporting criteria Modeling and usage were conducted in accordance with guidelines and quality criteria for deep learning research. 30 , 31 To protect patient privacy, models that were additionally trained on sensitive clinical operations data were not published online. 37 All models were developed as research tools and should not be used for decision-making in individual clinical cases. We followed eight recommended guidelines for reporting deep learning studies: Data sources: All data sources are outlined in Supplementary Table S1 . Data preprocessing: Raw, unadjusted scores were used. Symptom inventories with one or more scores missing were excluded. Partitioning: The same 50/50 train-test splitting of participants was performed for all models. Disjointness: Participant data was fully disjoint to avoid duplicates across training and test data. Any repeated measurements over time for the same participants of the same inventory were dropped. When learning to convert numeric scores from one inventory to another, only one inventory type was permitted in the training data for all participants at a time, and only one test-inventory item was permitted as the target. Models and Training : Four deep learning models were evaluated, each pretrained on different corpuses of general and medical text: (a) For the base model ( MiniLMBERT ), a pretrained Bidirectional Encoder Representations from Transformers (BERT) model was used. MiniLMBERT distilled the self-attention module of the last transformer layer of a large transformer 38 trained on several million sentence pairs. 39 This model is publicly available online. 40 Three other models pretrained on biomedical and clinical text were also evaluated: (b) ClinicalCovidBERT: A publicly available model pretrained on the CORD-19 medical dataset. 41 (c) VAClinicalDocsBERT: A clinically trained model that used ClinicalCovidBERT as a base, but with additional pretraining on 5,00,000 generic clinical operational documents from the Veterans Affair (VA) Corporate Data Warehouse (CDW 42 ), including admissions and discharge summaries. (d) VAMetadataBERT: A medically trained model that used ClinicalCovidBERT as a base, with additional training on 1.5 million text strings of clinical lab names, medication names, and document titles from the VA CDW. Additionally, we included a traditional bag-of-words vector model using term frequency—inverse document frequency weighted vectors in order to compare conventional word counting schema against the semantic models. Hyperparameters and Tuning: To improve reproducibility, no fine-tuning was performed. We did not change any of the weights of models at any stage to tune to symptom inventory content. Model Selection: The four BERT models were evaluated by comparing their relative performance at the task of correctly converting scores across inventories. Given scores on one set of inventory items, the task was to estimate all scores on another, fully distinct, set of inventory items. The ground truth for this problem was two dually administered inventories per person. Performance was only evaluated on held-out test participants. Model Metrics. The primary metric used in study was cosine similarity, S, which measures the semantic similarity of two symptom descriptions in the range 0–1. A value of 0 indicates the symptom descriptions share no meaningful similarity, while 1 means they have identical meanings. Given two output feature vectors A and B, cosine similarity is defined as the dot product of the two vectors divided by the product of their vector magnitudes, C = A·B/(|A‖B|). Higher cosine similarity means the relative angle between the vectors is closer in the feature space. Model prediction performance was measured using mean absolute error (MAE), binary accuracy, and multinomial accuracy. Multinomial accuracy was defined as the percentage of estimated scores that equaled the correct symptom severity on a 5-point Likert scale. Although accuracies near 50% normally indicate poor performance, the random guess accuracy was 20% for accuracy on a 5-point scale. Multinomial accuracy was our preferred metric since it measures accuracy on the true scale, and also strongly correlates with MAE (see Supplementary Fig. S1 ). Crosswalk model A “crosswalk” refers to the process of relating items of one inventory to another. A conceptual overview of the model process is shown in Figure 1 . First, the pretrained BERT transformer scored the similarity of symptom descriptions across inventories ( Fig. 1A ). For each item, its most similar item descriptions were found on other inventories. FIG. 1. Open in a new tab Flowchart diagram of the STS crosswalk model. (A) Symptom text descriptions were embedded using a semantic model, and the similarity of description pairs was quantified across inventories. (B) The most similar items across inventories were identified. (C) Empirical differences in the response distribution of pairs were corrected using a stochastic sampling approach. (D) Model estimated and actual inventory scores were compared across dually administered inventories. STS, semantic textual similarity. The second step adjusted for differences in scale response across items ( Fig. 1B ). For example, on the BSI-18, “Mild” is defined as the second option on the 5-point Likert scale. In contrast, “Mild” is the third point on the Likert scale for the RPQ. Therefore, different raw scores imply the same symptom severity level. A percentile sampling approach was used to mitigate these differences which accurately captured empirical score distributions (see Supplementary Fig. S2 ). It is essential to capture the natural variance of scores found in real world data for the converted scores to have practical utility, particularly for implementing clinical cutoffs. Therefore, the model outputs were explicitly forced to be stochastic in ways that mimic the natural score variations found in real data. If items had no single close analogue on other inventories, multiple items were used for prediction ( Fig. 1C ). After model construction, performance was assessed by comparing estimated and actual inventory scores for dually administered assessments ( Fig. 1D ). Statistical analysis Analysis was performed in Python 3. Validation data required no covariate adjustment, as the same set of individuals were dually administered the same two inventories. Crossvalidation with 50/50 test train splitting was used to determine statistical variation of results. The sentence-transformers 40 Python package was used for text embedding, while the statsmodels 43 and scikit-learn packages 44 were used to construct linear and machine learning models, respectively. Specifically, we implemented three comparison models (OLS [ordinary least squares regression], RF [random forest], GB [gradient boosting]). The number of tree estimators was set to 200 for GB and RF to improve performance. Confirmatory factor analysis was implemented using the factor-analyzer package. For prediction, we define A and B to be two dually administered assessments. The goal was to predict a single item in B, named b, given all items in A . Various strategies were explored to predict scores across inventories, including OLS, 1-NN (nearest neighbor), 3-NN, 5-NN, and a weighted approach that used all variables and the STS similarity scores as weights of relative item importance in the model. Overall, a 1-NN approach produced superior results on the test data. The nearest-neighbor approach selected only the most semantically similar item in A , a = argmax[ S(A,b) ], to predict b , excluding all other scores. The model did not use any score frequency or distribution information to link items. Results Data summary Table 1 summarizes characteristics of the cohort ( n = 6,607). The cohort showed good representation across age, sex, education level, race, ethnicity, and TBI status. In terms of injury severity, 1,159 participants (17.5%) were controls with no history of TBI, 5,400 participants (81.7%) had a history of mild TBI, and just 48 participants (0.7%) had a history of moderate/severe TBI. Across all participants, the median age was 29 years old, with an interquartile range of 20–43 years, and 29.4% were female. Using appropriate scoring schema, the means (and standard deviations) of total scores were BSI-18: 8.86 (10.52), RPQ: 17.49 (14.83), SCL-90-R: 70.5 (67.21), and NSI: 25.5 (16.95). An overlapping sample of the same participants ( n = 2,056) was administered both the BSI and RPQ as part of the same site visit assessment; these included 286 controls. Table 1. Descriptive Characteristics for the Cohort by Measure BSI-18 a RPQ a SCL-90-R NSI Sample size ( n ) 4,286 2,153 244 1,984 Age (years): Mean (std) 29.3 (15.3) 39.4 (16.3) 59.3 (15.9) 39.0 (9.7) >65 years 4.0% 8.1% 54.9% 0.7% Sex Male 63.1% 66.2% 89.3% 83.7% Female 36.9% 33.8% 10.7% 16.3% Education High school or less 2.9% 5.6% 3.7% 0.3% Some college 80.5% 61% 50% 59.4% Graduate degree 16.6% 33.4% 46.3% 40.3% Race Black 19.1% 16.3% 7% 18.4% White 77.1% 78.7% 91.8% 72.5% Other 3.8% 5.0% 1.2% 9.1% Ethnicity Hispanic/Latino 14.3% 19.7% 2.5% 16.5% History of TBI of any severity 88.3% 86.7% 57.0% 75.2% Measure q (Std) 8.9 (10.5) 17.5 (14.8) 70.5 (67.2) 25.5 (17.0) Open in a new tab a Indicates overlapping n = 2,056 administration. BSI-18, Brief Symptom Inventory–18; NSI, Neurobehavioral Symptom Inventory; RPQ, Rivermead Post-Concussion Symptoms Questionnaire; SCL-90-R, Symptom Checklist-90-Revised; TBI, traumatic brain injury. Semantic textual similarity Initial analysis was performed using the general language model, MiniMLBERT. Illustrative examples of MiniMLBERT symptom similarities are shown in Table 2 for symptom pair comparisons. Figure 2 shows a stem plot of cosine similarities for one RPQ symptom, “Nausea and/or vomiting,” compared with all BSI-18 symptom descriptions. Most of the 18 BSI-18 items were classified as conceptually unrelated to the item, but “Nausea or upset stomach” was strongly related. As these two items had maximum similarity, they formed one cross-inventory pair. Conversely, the model did not link unrelated items, even if they contained overlapping words (see Supplementary Data S2 ). Table 2. Example Comparisons of Symptoms Ranked by Semantic Similarity Using Two Methods: TF-IDF and STS Symptom text # 1 Symptom text # 2 TF-IDF cosine similarity STS cosine similarity Numbness or tingling on parts of my body Numbness or tingling in parts of body 0.95 0.94 Feeling dizzy Faintness or dizziness 0.31 0.76 Nausea Upset stomach 0.01 0.72 Feeling shy or uneasy with the opposite sex Nervousness or shakiness inside 0.14 0.55 Feeling nervous when you are left alone Suddenly scared for no reason 0.20 0.45 Sensitivity to light or sound Feeling blue 0.00 0.32 Poor coordination Feeling weak in parts of your body 0.01 0.31 Trouble remembering things Trouble getting your breath 0.26 0.20 Blaming yourself for things Change in taste and/or smell 0.12 0.03 Open in a new tab Small changes have been made to protect instrumental integrity. Overlapping words are shown in bold. Unlike the TF-IDF cosine similarity scores, the STS cosine similarity scores robustly mirror symptom pair relatedness. STS, semantic textual similarity; TF-IDF, term frequency-inverse document frequency. FIG. 2. Open in a new tab Semantic textual similarity (STS) of symptoms. A stem plot shows the cosine similarity of the RPQ symptom “Nausea and/or vomiting” with the 18 symptoms assessed by the BSI-18. BSI-18, Brief Symptom Inventory–18; RPQ, Rivermead Post-Concussion Symptoms Questionnaire. To extend beyond single examples, Figure 3 shows the similarity scores of all items across all inventories (NSI, SCL-90-R, BSI-18, and RPQ), sorted and color-coded by cross-inventory comparison. The SCL→BSI comparison had 18 near-identical item pairs (cyan stars, top). This means that the BSI-18 was effectively a semantic subset of the SCL-90-R, as would be expected given the BSI-18 uses items from the SCL-90-R. In this way, STS can rapidly screen for closely related items across inventories, regardless of whether these relationships are established or not in the literature. FIG. 3. Open in a new tab Similarity of symptom pairs across all inventories. The semantic textual similarities for all items across the NSI, SCL-90-R, BSI-18, and RPQ are shown sorted and color-coded by inventory pair. Semantically related items are embedded in a large background of unrelated symptomatology; for example, the BSI-18 is a subset of the SCL-90-R, so there are 18 near-identical SCL-BSI similarity scores (cyan stars, top). BSI-18, Brief Symptom Inventory–18; NSI, Neurobehavioral Symptom Inventory; RPQ, Rivermead Post-Concussion Symptoms Questionnaire; SCL-90-R, Symptom Checklist-90-Revised. To assess the semantic similarity of different inventories in aggregate, the distribution of closest-pair cosine similarities was found for each inventory pair ( Fig. 4 ). Overall, 41.7% of the closest pairs were S > 0.6, indicating that many single symptoms had close analogues in other inventories. When considering aggregate similarity across inventories, directionality matters (i.e., A→B vs. B→A). The NSI and RPQ, both TBI-related inventories, contained similar content, whereas the NSI and BSI-18 showed relatively low median similarity. FIG. 4. Open in a new tab Quantifying the similarity of symptom inventories. Boxplots show the distribution of maximum semantic textual similarities of symptoms broken out for each inventory crosswalk in both directions. Overall, 5 of the 12 inventory comparisons showed maximum linked text similarity medians of 0.6 or higher. BSI-18, Brief Symptom Inventory–18; NSI, Neurobehavioral Symptom Inventory; RPQ, Rivermead Post-Concussion Symptoms Questionnaire; SCL-90-R, Symptom Checklist-90-Revised. Inventory score prediction Before implementing cross-inventory models, within-inventory score prediction was implemented to assess the ability to convert scores in the absence of cross-inventory effects ( Supplementary Fig. S3 ). Within-inventory models used data for all items in a given inventory (except one) as explanatory variables to estimate scores on the single, reserved item. As these experiments were not subject to any cross-inventory effects (e.g., differences in administration or scoring), they set a plausible limit on the accuracy free of cross-inventory effects. The average prediction accuracy was 57.2% for within-inventory estimation of BSI-18 and RPQ. Figure 5 shows the cross -inventory prediction accuracies for four different models: (1) STS, (2) linear regression, (3) random forest, and (4) gradient boosting. The data included n = 2,056 individuals who were administered both the RPQ and BSI-18, split randomly into 50/50 test-train groups (designating n = 1,028 held out test participants). Only RPQ items were used to estimate BSI item scores (RPQ→BSI-18, gray circles), and only BSI-18 items used to estimate RPQ item scores (BSI-18→RPQ, white circles). The STS model (blue line) achieved 54.3% multinomial accuracy when predicting scores across inventories, consistently outperforming the benchmark models across a wide range of symptoms, and reaching close to the average expected accuracy on within-inventory score estimation when no cross inventory effects or related processes are required (57.2%). FIG. 5. Open in a new tab Cross-inventory symptom prediction. Cross-inventory accuracies are shown for the semantic textual similarity (STS) model, alongside three comparison models. Predictions were tested with 50/50 test-train splitting of n = 2,056 subjects who were all dually administered both the RPQ and BSI. Only RPQ items were used to estimate BSI item scores (RPQ→BSI, right gray circles), and only BSI items were used to estimate RPQ item scores (BSI→RPQ, right white circles). The nonclinical semantic model (blue line) achieved 54.3%, significantly ( p < 0.001) outperforming all other benchmark approaches. Within measure OLS score prediction accuracies are shown as gray bars. GB, gradient boosting; OLS, ordinary least squares regression; RF, random forest. Model performance Benchmark models were evaluated by comparing their relative performance at cross-inventory symptom score conversion for five different multinomial and binary classification tasks ( Table 3 ). Table 3. Cross-Inventory Model Accuracies for Right Models Model and accuracy t > 0 t > 1 t > 2 t > 3 Multinomial Average Linear regression 71.0% 80.2% 87.9% 95.1% 48.0% 74.4% Gradient boosting 63.2% 76.4% 86.6% 94.9% 40.6% 69.4% Random forest 70.2% 78.9% 87.6% 95.0% 47.0% 73.6% TD-IDF 58.5% 67.3% 82.6% 93.4% 45.2% 69.4% MiniLMBERT 71.2% 76.9% 85.0% 92.8% 54.3% 74.8% VAMetadataBERT 69.2% 75.0% 83.4% 92.5% 52.7% 73.3% ClinicalCovidBERT 70.0% 75.6% 84.1% 92.7% 53.2% 73.8% VAClinicalDocsBERT 69.0% 74.7% 83.4% 92.3% 52.0% 72.9% Open in a new tab Four benchmark models, linear regression, gradient boosting, random forest, and term frequency−inverse document frequency (TF-IDF), and four pretrained variants of semantic models were evaluated for accuracy under different scenarios. The models are assessed under five different scenarios: Prediction of binary classification at all four possible threshold scores (0, 1, 2, 3), and multinomial accuracy. Predictions were tested with 50/50 test-train splitting of n = 2,056 subjects who were all dually administered both the RPQ and BSI. The model trained on generic, nonmedical text ( MiniLMBERT ) performed with the highest accuracy overall (74.8%), although linear regression performed optimally at some thresholds. MiniLMBERT also performed optimally on the clinically useful multinomial metric, which does not presuppose a specific score threshold ( t ). BSI-18, Brief Symptom Inventory–18; RPQ, Rivermead Post-Concussion Symptoms Questionnaire; TF-IDF, term frequency-inverse document frequency. Maximum scores shown in bold for the multinomial and average metrics. Overall, the MiniLMBERT model trained on a general corpus of text showed the highest accuracy, 74.8%, when averaged across all scenarios, consistently outperforming the three clinically pretrained transformers (72.9–73.8%). MiniLMBERT also achieved 54.3% on the more challenging multinomial prediction task, outperforming all benchmark models (40.6–48.0%), and the three other medically pretrained transformers (52.0–53.2%). These results persisted under 30-fold 50/50 test/train splitting, and a large effect ( d = 27.73) between MiniLMBERT performance and the best-performing non-STS (OLS) model was found under repetition ( Supplementary Fig. S2 ). Correlation and factor analysis found the properties of the measure scales and subscales were broadly preserved under conversion (see Supplementary Data S3 : Correlation and Confirmatory Factor Analysis). The symptom inventory conversion tool is available as a web interface (see Supplementary Fig. S4 ). To explore whether model efficacy varied across sex, MiniLMBERT performance was stratified for dually administered male ( N = 1,349) and female ( N = 707) participants. The model predicted female symptoms with 6% lower accuracy than male symptoms, equivalent to an effect size of d = −0.43 ( p < 0.001). Relatedly, only 14.7% of the training cohort were female. An age-stratified performance evaluation was conducted for two groups of dually administered participants: aged 65 years or above ( N = 190) and aged below 65 years ( N = 1,866). Interestingly, symptoms were more accurately predicted for the elderly group (61.7%) than for those below 65 years of age (53.4%, p < 0.001). Discussion Providers often use standard inventories for initial evaluation and tracking of TBI symptoms. However, comparing results across distinct symptom inventories is challenging due to subtle differences in how symptoms are described, assessed, and conceptualized. These differences confound the aggregation of data and findings across clinical and research settings and limit comparison between historical and current studies. To address this problem, this study reported a novel application of language models to rapidly and accurately link items and scores across self-reported symptom inventories. We tested four semantic symptom-linking models on data for thousands of individuals who each completed two different symptom inventories. Overall, a deep learning model trained on a general text corpus showed the highest accuracy, which confirmed our hypothesis. The superior performance of the generic language model over clinically pretrained models is consistent with the straightforward language inventories used to describe symptoms. Although the general text semantic model showed the best performance overall, model performance varied considerably across individual items. One issue when comparing symptom inventories is that their content can overlap. For example the BSI-18 is a short version contained in the SCL-90. Initially considered a potential challenge for the work, the existence of direct analogues across inventories was valuable because they provided identical semantic ground truths across inventories. This facilitated the direct observation of the effects of different inventory scoring schemes and scales for otherwise identical items. This study also offers useful insights into the nature of TBI-related symptomatology and measurement. Many studies conduct multiple inventory assessments to more completely capture a wide range of potential patient experiences. Semantic insights could help to guide and optimize the selection of complementary instruments. Across inventories, about two-thirds of the NSI and RPQ symptoms were strongly related, confirming that they were semantically similar assessments, which was anticipated since both assess TBI. By contrast, those wishing to pair a general and TBI specific inventory could consider the NSI and BSI-18, as they had lower average similarity than other inventory pairs. Beyond existing inventories, deep learning text similarity paradigms might also be able to assist in the development of new, abbreviated questionnaires that more precisely assess distress, and one could imagine a synthetic superscale that draws semantically from all inventories. Although “harmonization” commonly refers to data aggregation and cleaning, true data harmonization aims to minimize unwanted measurement variations while preserving the underlying meaning of the measures of interest. The central goal of this study was to use harmonization techniques to produce online resources that can link scores across previously incompatible inventories. However, in the course of developing a deep learning pipeline for cross-walking across symptom inventories, we observed that the STS model did not always link items with the highest empirical correlation on scores. Instead, it detected and leveraged subtle relationships between symptom phrasings and, in doing so, exceeded the performance of other empirically trained linear and machine learning models. The current finding that the similarity of text describing symptoms was generally more useful than training on empirical data is surprising. One explanation for why transformer models leveraging semantic similarity have an advantage over shallow baselines such as bag-of-words is because the diversity of terms used in clinical text is large but quite general. This is consistent with recent reports where a majority of top-ranking systems make use of transfer learning to outperform traditional baselines. 45 Many studies using AI in medicine have demonstrated impressive gains in diagnostic accuracy, but the diagnostic labels needed to train machine learning models are often assigned using clinical evaluation tools with long-standing data compatibility issues. This is doubly true in TBI research where case definitions are often unclear. This study leveraged AI to address a fundamental decision-focused task—the harmonization of clinical measurements. If deep learning can be used to improve the quality of tools that assign training data labels, then it may be possible to achieve further, untapped gains in accuracy across a range of health-related learning tasks. Strengths and limitations Strengths of this study include a large, aggregated sample drawn from 16 data sources, high-quality dually administered test data, evaluation of multiple models trained on different medical and general text sources, and detailed investigations of both disease-specific and general symptom inventories. Other strengths include a close collaboration between methodologists and clinical experts that ensured patient safety, privacy, and careful adherence to recommended AI reporting criteria, and both data and code are made available. There are some limitations of the current study worth noting. First, three or more inventories per participant were unavailable. We also did not adjust for symptom validity. Three of the inventories use distinct reference time frames, and differences in administration were not considered. However, since the method was nearly as accurate when estimating scores across, compared with within, inventories, cross-assessment effects were largely mitigated. Fourth, the data were drawn from 18 English language sources, including both military and civilian datasets. Therefore, the findings may not generalize to specific populations. However, insofar as this was tested, stratifying the results by age and sex showed only modest variations in performance. Choice of model comes with relative benefits and costs, including potential loss of human interpretability. Simple models such linear regression afford clear coefficients and interpretability. Conversely, why STS models link certain item pairs and not others may be difficult to interpret. In some scenarios, more interpretable models may be advantageous, for example if explainability of the model process, rather than optimization of overall conversion accuracy is not the primary objective. Inventories do not capture all elements of personal experience, and some may even systematically screen out or inadequately capture meaning. The extent to which crosswalk tools incur related loss of information regarding the patient’s experience should be studied. The finite number of items available per instrument constrains the accuracy of crosswalks. To address this, future work could explore extensions using reinforcement learning through human feedback to learn an optimal process for handling items that have low semantic similarity. Future development of the model to specifically target underperforming item conversions could help to improve overall model performance. For example, a single item in an inventory can potentially correlate with multiple items in another, but some items may have no close analogue or correlating item on another instrument. Further work is needed to explore whether one-to-many relationships could resolve this issue using multivariate models that aggregate information from multiple weak correlated predictors. Only inventories with 5-point scales were used, but there is no reason why this approach could not be extended to inventories with different numeric scales. Nevertheless, this study utilized one of the largest samples of symptom inventory data in TBI yet assembled and focused upon the most widely used and recommended measures in the field. 46–48 Authors’ Contributions Original data were collected by M.T., M.R.N., R.A.M., D.F.T., W.S.C.W., and E.A.W. E.K., E.L.D., E.A.W., D.F.T., F.G.H., K.D.O., H.M.L., and S.W.L. conceived of the project. E.L.D. and H.M.L. compiled and cleaned the data. E.K., S.V., K.S.P., and R.A. completed the processing and analysis. E.K. wrote and revised the article, and all authors reviewed, edited, and approved the article. Data Availability We included 16 different sources of data in this study. Of these, 6 data sources are freely available online as part of the FITBIR data repository, hosted at https://fitbir.nih.gov . The other 10 datasets are available from the corresponding author upon reasonable request, pending IRB approval for dissemination of data, and additional institutional approval of data use and access. Code Availability Codes for STS modeling, symptom inventory conversion, and web tool design are available online at https://github.com/ShashanKV98/symptom-inventories-calculator . Disclaimer The views expressed in this article are those of the author(s) and do not reflect the official policy of the Department of Army/Navy/Air Force, Department of Defense, or U.S. government. Author Disclosure Statement None of the authors have competing interests relevant to this article. N.J. and P.M.T. received a research grant from Biogen, Inc., for research unrelated to this article. T.B.M. receives compensation as a member of the Clinical and Scientific Advisory Board for Quadrant Biosciences, Inc. V.N. holds a grant from Roche Pharmaceuticals for a project unrelated to this article. A.O. is a cofounder and owner of Nordic Brain Tech AS. Funding Information This work was supported by R61NS120249 to E.L.D., E.A.W., F.G.H., and D.F.T. Supplementary Data S1 neu.2024.0301_Supp_DataS1.pdf (82.6KB, pdf) Supplementary Data S2 neu.2024.0301_Supp_DataS2.pdf (70.9KB, pdf) Supplementary Data S3 neu.2024.0301_Supp_DataS3.pdf (133.9KB, pdf) Supplementary Figure S1 neu.2024.0301_Supp_FigS1.pdf (297.8KB, pdf) Supplementary Figure S2 neu.2024.0301_Supp_FigS2.pdf (121.2KB, pdf) Supplementary Figure S3 neu.2024.0301_Supp_FigS3.pdf (195.7KB, pdf) Supplementary Figure S4 neu.2024.0301_Supp_FigS4.pdf (219.4KB, pdf) Supplementary Table S1 neu.2024.0301_Supp_TableS1.pdf (60.9KB, pdf) Supplementary Table S2 neu.2024.0301_Supp_TableS2.pdf (82.1KB, pdf) References 1. Wallace W, Chan C, Chidambaram S, et al. The diagnostic and triage accuracy of digital and online symptom checker tools: A systematic review. NPJ Digit Med 2022;5(1):118; doi: 10.1038/s41746-022-00667-w [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 2. King NS, Crawford S, Wenden FJ, et al. The Rivermead Post-Concussion Symptoms Questionnaire: A measure of symptoms commonly experienced after head injury and its reliability. J Neurol 1995;242(9):587–592; doi: 10.1007/BF00868811 [ DOI ] [ PubMed ] [ Google Scholar ] 3. Potter S, Leigh E, Wade D, et al. The Rivermead Post-Concussion Symptoms Questionnaire: A confirmatory factor analysis. J Neurol 2006;253(12):1603–1614; doi: 10.1007/s00415-006-0275-z [ DOI ] [ PubMed ] [ Google Scholar ] 4. Cicerone KD, Kalmar K. Persistent postconcussion syndrome: The structure of subjective complaints after mild traumatic brain injury. J Head Trauma Rehabil 1995;10(3):1–17; doi: 10.1097/00001199-199510030-00002 [ DOI ] [ Google Scholar ] 5. Derogatis LR. Brief Symptom Inventory 18. n.d. Available from: https://www.researchgate.net/profile/Rachel-Bachner-Melman/post/Can_the_SCL-10R_be_used_to_as sess_risk_for_multiple_mental_health_conditions/attachment/5fbfa50b2f079e0001be051a/AS%3A9621 05134022658%401606395147689/download/bsi+info.docx [Last accessed: July 27, 2023]. 6. Derogatis LR, Unger R. Symptom Checklist-90-Revised. The Corsini Encyclopedia of Psychology; 2010; doi: 10.1002/9780470479216.corpsy0970 [ DOI ] [ Google Scholar ] 7. Maas AIR, Menon DK, Manley GT, et al. InTBIR Participants and Investigators . Traumatic brain injury: Progress and challenges in prevention, clinical care, and research. Lancet Neurol 2022;21(11):1004–1060; doi: 10.1016/S1474-4422(22)00309-X [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 8. Karaliute M, Saksvik SB, Smevik H, et al. Methodology matters: Comparing approaches for defining persistent symptoms after mild traumatic brain injury. Neurotrauma Rep 2021;2(1):603–617; doi: 10.1089/neur.2021.0028 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 9. Lempke LB, Schmidt JD, Lynall RC. Athletic trainers’ concussion-assessment and concussion-management practices: An update. J Athl Train 2020;55(1):17–26; doi: 10.4085/1062-6050-322-18 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 10. Rosenbusch H, Wanders F, Pit IL. The semantic scale network: An online tool to detect semantic overlap of psychological scales and prevent scale redundancies. Psychol Methods 2020;25(3):380–392; doi: 10.1037/met0000244 [ DOI ] [ PubMed ] [ Google Scholar ] 11. Scollard P, Choi S-E, Lee ML, et al. Ceiling effects and differential measurement precision across calibrated cognitive scores in the Framingham Study. Neuropsychology 2023;37(4):383–397; doi: 10.1037/neu0000828 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 12. Nance RM, Delaney JAC, Golin CE, et al. Co-calibration of two self-reported measures of adherence to antiretroviral therapy. AIDS Care 2017;29(4):464–468; doi: 10.1080/09540121.2016.1263721 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 13. Mukherjee S, Choi S-E, Lee ML, et al. Cognitive domain harmonization and cocalibration in studies of older adults. Neuropsychology 2023;37(4):409–423; doi: 10.1037/neu0000835 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 14. Boccardi M, Monsch AU, Ferrari C, et al. Consortium for the Harmonization of Neuropsychological Assessment for Neurocognitive Disorders (https://nextcloud.dzne.de/index.php/s/EwXjLab9caQTbQe) . Harmonizing neuropsychological assessment for mild neurocognitive disorders in Europe. Alzheimers Dement 2022;18(1):29–42; doi: 10.1002/alz.12365 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 15. Pinkham AE, Penn DL, Green MF, et al. The social cognition psychometric evaluation study: Results of the expert survey and RAND panel. Schizophr Bull 2014;40(4):813–823; doi: 10.1093/schbul/sbt081 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 16. Dams-O’Connor K, Awwad HO, Hoffman S, et al. Alzheimer’s disease-related dementias summit 2022: National research priorities for the investigation of post-traumatic brain injury Alzheimer’s disease and related dementias. J Neurotrauma 2023;40(15–16):1512–1523; doi: 10.1089/neu.2022.0514 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 17. Meeuws S, Yue JK, Huijben JA, et al. Common data elements: Critical assessment of harmonization between current multi-center traumatic brain injury studies. J Neurotrauma 2020;37(11):1283–1290; doi: 10.1089/neu.2019.6867 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 18. Polinder S, Haagsma JA, van Klaveren D, et al. Health-related quality of life after TBI: A systematic review of study design, instruments, measurement properties, and outcome. Popul Health Metr 2015;13:4; doi: 10.1186/s12963-015-0037-1 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 19. Brady KJS, Ni P, Carlasare L, et al. Establishing crosswalks between common measures of burnout in US physicians. J Gen Intern Med 2022;37(4):777–784; doi: 10.1007/s11606-021-06661-4 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 20. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) IEEE; 2016; pp. 770–778; doi: 10.1109/cvpr.2016.90 [ DOI ] [ Google Scholar ] 21. Lu B, Li H-X, Chang Z-K, et al. A practical Alzheimer’s disease classifier via brain imaging-based deep learning on 85,721 samples. J Big Data 2022;9(1):1–22; doi: 10.1186/s40537-022-00650-y [ DOI ] [ Google Scholar ] 22. Liu Y, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv [csCL] 2019. 23. Wang A, Singh A, Michael J, et al. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. arXiv [csCL] 2018. 24. Rajpurkar P, Chen E, Banerjee O, et al. AI in health and medicine. Nat Med 2022;28(1):31–38; doi: 10.1038/s41591-021-01614-0 [ DOI ] [ PubMed ] [ Google Scholar ] 25. He Y, Chen J, Antonyrajah D, et al. BERTMap: A BERT-Based Ontology Alignment System. AAAI 2022;36(5):5684–5691; doi: 10.1609/aaai.v36i5.20510 [ DOI ] [ Google Scholar ] 26. Gonçalves RS, Kamdar MR, Musen MA. Aligning Biomedical Metadata with Ontologies Using Clustering and Embeddings. The Semantic Web Springer International Publishing; 2019; pp. 146–161; doi: 10.1007/978-3-030-21348-0_10 [ DOI ] [ Google Scholar ] 27. Bajaj G, Nguyen V, Wijesiriwardene T, et al. Evaluating Biomedical BERT Models for Vocabulary Alignment at Scale in the UMLS Metathesaurus. arXiv [csCL] 2021. [ DOI ] [ PMC free article ] [ PubMed ] 28. Chandrasekaran D, Mago V. Evolution of semantic similarity—a survey. ACM Comput Surv 2022;54(2):1–37; doi: 10.1145/3440755 [ DOI ] [ Google Scholar ] 29. Prince C, Bruhns ME. Evaluation and treatment of mild traumatic brain injury: The role of neuropsychology. Brain Sci 2017;7(8); doi: 10.3390/brainsci7080105 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 30. Mongan J, Moy L, Kahn CE. Jr., Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A guide for authors and reviewers. Radiol Artif Intell 2020;2(2):e200029; doi: 10.1148/ryai.2020200029 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 31. Roberts M, Driggs D, Thorpe M, et al. AIX-COVNET . Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nat Mach Intell 2021;3(3):199–217; doi: 10.1038/s42256-021-00307-0 [ DOI ] [ Google Scholar ] 32. Eisenhauer JG. Meta‐analysis and mega‐analysis: A simple introduction. Teach Stat 2021;43(1):21–27; doi: 10.1111/test.12242 [ DOI ] [ Google Scholar ] 33. Dennis EL, Baron D, Bartnik‐Olson B, et al. ENIGMA brain injury: Framework, challenges, and opportunities. Hum Brain Mapp 2022;43(1):149–166; doi: 10.1002/hbm.25046 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 34. Wilde EA, Dennis EL, Tate DF. The ENIGMA brain injury working group: Approach, challenges, and potential benefits. Brain Imaging Behav 2019. Under Review(Special Issue). [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 35. Cifu DX, Dixon KJ. Chronic effects of neurotrauma consortium. Brain Inj 2016;30(12):1397–1398; doi: 10.1080/02699052.2016.1219065 [ DOI ] [ PubMed ] [ Google Scholar ] 36. Anonymous. Home. n.d. Available from: https://fitbir.nih.gov/ [Last accessed: August 29, 2023]. 37. Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med 2023;6(1):120; doi: 10.1038/s41746-023-00873-0 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 38. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Adv Neural Inf Process Syst 2017;30. [ Google Scholar ] 39. Wang W, Wei F, Dong L, et al. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. arXiv [csCL] 2020. 40. Reimers N, Gurevych I. Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. arXiv [csCL] 2019. 41. Chapman AB, Peterson KS, Turano A, et al. A Natural Language Processing System for National COVID-19 Surveillance in the US Department of Veterans Affairs. In: Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. [ Google Scholar ] 42. Anonymous. Health Services Research & Development. n.d. Available from: https://www.hsrd.research.va.gov/ [Last accessed: September 1, 2023]. 43. Seabold S, Perktold J. Econometric and statistical modeling with Python skipper seabold 1 1. Proc 9th Python Sci Conf n.d. [ Google Scholar ] 44. Pedregosa F, Varoquaux G, Gramfort A, et al. Scikit-Learn: Machine learning in python. arXiv [csLG] 2012;2825–2830. 45. Wang Y, Fu S, Shen F, et al. The 2019 n2c2/OHNLP track on clinical semantic textual similarity: Overview. JMIR Med Inform 2020;8(11):e23375; doi: 10.2196/23375 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 46. Wilde EA, Whiteneck GG, Bogner J, et al. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil 2010;91(11):1650–1660.e17; doi: 10.1016/j.apmr.2010.06.033 [ DOI ] [ PubMed ] [ Google Scholar ] 47. Hicks R, Giacino J, Harrison-Felix C, et al. Progress in developing common data elements for traumatic brain injury research: Version two–the end of the beginning. J Neurotrauma 2013;30(22):1852–1861; doi: 10.1089/neu.2013.2938 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] 48. Broglio SP, Kontos AP, Levin H, et al. National Institute of Neurological Disorders and Stroke and Department of Defense sport-related concussion common data elements version 1.0 recommendations. J Neurotrauma 2018;35(23):2776–2783; doi: 10.1089/neu.2018.5643 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials Supplementary Data S1 neu.2024.0301_Supp_DataS1.pdf (82.6KB, pdf) Supplementary Data S2 neu.2024.0301_Supp_DataS2.pdf (70.9KB, pdf) Supplementary Data S3 neu.2024.0301_Supp_DataS3.pdf (133.9KB, pdf) Supplementary Figure S1 neu.2024.0301_Supp_FigS1.pdf (297.8KB, pdf) Supplementary Figure S2 neu.2024.0301_Supp_FigS2.pdf (121.2KB, pdf) Supplementary Figure S3 neu.2024.0301_Supp_FigS3.pdf (195.7KB, pdf) Supplementary Figure S4 neu.2024.0301_Supp_FigS4.pdf (219.4KB, pdf) Supplementary Table S1 neu.2024.0301_Supp_TableS1.pdf (60.9KB, pdf) Supplementary Table S2 neu.2024.0301_Supp_TableS2.pdf (82.1KB, pdf) Data Availability Statement We included 16 different sources of data in this study. Of these, 6 data sources are freely available online as part of the FITBIR data repository, hosted at https://fitbir.nih.gov . The other 10 datasets are available from the corresponding author upon reasonable request, pending IRB approval for dissemination of data, and additional institutional approval of data use and access. Articles from Journal of Neurotrauma are provided here courtesy of SAGE Publications ACTIONS View on publisher site PDF (1.4 MB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• One Size Does Not Fit All: A Data-Driven Framework for Personalized Comorbidity Scoring#417387
• Symptom burden in multiple long-term conditions: An AI-supported, mixed-methods concept elicitation study.#378365
• Symptom Burden: A Concept Analysis.#214143
• The Effect of Extended Yellow Fever Vaccine Information on Symptom Reports Following Vaccination#839512
• Towards Healthy Aging through Semantic Enrichment.#322792