Utilizing natural language processing to analyze student narrative reflections for medical curriculum improvement.

Utilizing natural language processing to analyze student narrative reflections for medical curriculum improvement - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice Med Teach . Author manuscript; available in PMC: 2026 Jun 1. Published in final edited form as: Med Teach. 2024 Aug 20;47(6):970–975. doi: 10.1080/0142159X.2024.2390034 Utilizing natural language processing to analyze student narrative reflections for medical curriculum improvement Amy L Olex a , Adam Garber b , Sally A Santen c , Courtney Blondino d , Stephanie Goldberg e , Deborah DiazGranados a, f Author information Article notes Copyright and License information a C. Kenneth and Diane Wright Regional Center for Clinical and Translational Science, Virginia Commonwealth University, Richmond, Virginia, USA b Department of Internal Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, USA c Department of Emergency Medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia, USA d Department of Health Sciences, University of Richmond, Richmond, Virginia, USA e Department of Surgery at Virginia Commonwealth University School of Medicine, Richmond, Virginia, USA f Department of Psychiatry, Virginia Commonwealth University School of Medicine, Richmond, Virginia, USA ✉ Correspondence should be addressed to Deborah DiazGranados, PhD, Department of Psychiatry, School of Medicine, Virginia Commonwealth University, PO Box 980466, Richmond, VA 23298; diazgranados@vcu.edu Issue date 2025 Jun. PMC Copyright notice PMCID: PMC11949084  NIHMSID: NIHMS2015653  PMID: 39161978 The publisher's version of this article is available at Med Teach Abstract Motivation: Medical curricula improvement is an ongoing process to keep material relevant and improve the student’s learning experience to better prepare them for patient care. Many programs utilize end-of-year evaluations, but these frequently have low response rates and lack actionable feedback. We hypothesized that student reflections written during a fourth year Sub-Internship could be used retrospectively to mine additional information as feedback for future curriculum adjustments. However, reflections contain a large amount of narrative content that would require a cumbersome and essentially infeasible manual review process for busy medical education faculty. Methods: We developed a Natural Language Processing (NLP) pipeline to automatically identify common themes and topics present in the set of reflective writings that could be used to improve the curriculum. The dataset contains required responses to a faculty issued question submitted between August 2016 and July 2018 about challenges experienced during the medical students fourth year Sub-Internship. Results: Eleven distinct topics were identified, with several being subsequently addressed in future iterations of the curriculum. Conclusion: Utilizing NLP on reflective writings was able to identify areas of curriculum improvement, and the NLP results provided a quick and easy way to explore the main themes and challenges expressed by students. Keywords: natural language processing, curriculum assessment and evaluation, student feedback Introduction Actionable items for course improvement are generally obtained through end-of-year feedback from students, which often have low response rates and limited actionable feedback for the course instructors. For this work we used reflective writings from fourth-year medical students, written as part of their Sub-Internship (Sub-I) rotation, to identify actionable items for curriculum improvements. Reflection plays a critical role in the learning cycle: synthesizing learning and discovering connections within an educational experience ( Mann, Gordon, and MacLeod 2009 ). They help students process information and experiences, develop self-awareness, and deepen understanding of complex concepts ( Winkel et al. 2017 ). Reflective writing datasets contain a large amount of narrative content, the processing of which is a time and resource intensive commitment that poses a significant challenge to clinician educators. To address this issue we developed a Natural Language Processing (NLP) pipeline ( Olex et al. 2020 ) that automatically identifies common themes and topics present in reflective writings. NLP is a cornerstone of artificial intelligence that enables machines to process and interpret human language. In the field of medical education, NLP is being used to assess medical student competencies by automatically scoring essays and medical writings ( Dias, Gupta, and Yule 2019 ), evaluate medical training using clinical notes and problem-based learning transcripts ( Denny et al. 2015 ; Spickard et al. 2014 ), and track clinical exposure ( Chen et al. 2014 ). In this work we utilize NLP to analyze student narrative reflections from the Sub-I rotation to evaluate the curriculum from the perspective of the student, which is often not apparent through traditional end-of-year evaluations. We provide an overview of the NLP algorithm, analysis results of 665 medical student reflections spanning 2 years, how the results informed curriculum improvement, and a discussion of how to integrate NLP into a medical curriculum evaluation program. Methods Data Source, Study Population and Study Team In this study, we retrospectively analyzed reflective writing blog posts from the Sub-I rotation, which is the main clinical course of fourth-year medical school. Responses to the prompt “What challenges have you faced thus far in the Sub-Internship role? (for example, incorporating into the team, understanding your role, achieving goals and objectives, transitioning to the Sub-I role, etc).” were collected between August 2016 and July 2018 and manually cleaned to remove the initial prompt post, any direct quotes of the prompt embedded within a response, direct quotes of previous posts, and all double postings. The final data set, referred to as the Sub-I Corpus, included 665 posts from 651 students (average 32/rotation, 4 students participated twice). The study team included 3 medical education subject matter experts who provided guidance during algorithm development, identified stopwords, and reviewed, interpreted, and acted on the results, as well as a single NLP expert who implemented the algorithm, executed all data processing, including manual cleaning and NLP analysis, and provided guidance during the interpretation of results. NLP Analysis We developed an automated NLP pipeline to identify student challenges with the goal of finding actionable course feedback. This pipeline has been implemented in the self-service tool TopEx ( topex.cctr.vcu.edu ), which clusters sentences in a corpus by common themes. The TopEx pipeline includes four automated steps ( Figure 1 ): 1) Sentence Normalization, 2) Sentence Representation, 3) Sentence Clustering, and 4) Topic Analysis. Each of these are described briefly below, and in more detail in Appendix 1 . Only two manual data processing steps are required: 1) Data formatting and cleaning involves getting the corpus into the appropriate input format (e.g. text files or an Excel spreadsheet), removing empty responses, and any direct prompt quotes as described previously--a process that took approximately 15–30 minutes to complete; 2) Custom stopword identification, which is described below in the Sentence Normalization section. TopEx was previously validated on a subset of the Sub-I Corpus, which revealed that it performs well at identifying topics with distinct vocabularies (computer system issues) versus more interpretive topics (feeling overwhelmed)( Olex et al. 2020 ). This previous work includes technical details of the TopEx algorithm, the development and coding of a gold standard dataset for testing and evaluation, and a performance evaluation. Figure 1. Open in a new tab TopEx scatterplot. Each dot represents a sentence that was analyzed and clustered. Numbers refer to the cluster numbers listed in Tables 1 and 2 , and Supplementary Digital Appendix 4 . Sentences that are closer to each other in space generally discuss similar topics. See [6] for methods details. Sentence Normalization Normalization removes lexical variation , which is the variability in how concepts are expressed in text (e.g. “she’s running” vs “she is running”) so that downstream steps will be less affected by differences in the writing style of students. Uninformative words, referred to as stopwords , such as “the”, “and”, or “that”, are also automatically removed. The authors worked together, running TopEx in an iterative fashion, to identify a set of custom stopwords that included medical department abbreviations, ordinals, and other commonly used abbreviations that were utilized by many of the students and were not particularly helpful in identifying challenges ( Appendix 1 has more details and Appendix 2 provides the list of stopwords used in this work). Sentence Representation Normalized sentences are converted to a numerical vector , which is a representative list of numbers that encodes the semantics of a sentence such that sentences with similar semantics will be represented by mathematically similar numerical vectors. This is achieved by generating a Term Frequency-Inverse Document Frequency (TF-IDF) matrix ( Appendix 1 has additional details, Appendix 3 has a worked example). Sentence Clustering Numerical vectors are compared to each other using a distance metric where the smaller the value, the more numerically similar two vectors are. Clustering uses these values to group sentences together that are mathematically close to each other, which indicates those sentences are discussing a similar topic. Topic Analysis Latent Dirichlet Allocation (LDA)( Blei, Ng, and Jordan 2003 ) is used to identify main topics of each cluster and outputs a ranked list of relevant words from the included sentences as a topic summary. Medical educators manually reviewed the topics and sentences in each cluster to provide a human-generated summary and identify challenges that can be addressed in the curriculum ( Table 1 ). This process of summarizing and interpreting results from hundreds of AI responses took approximately 2 hours, which is exceedingly fast compared to the amount of time (i.e., days) necessary for reading, documenting and discussing student reflection statements that was required previously. Table 1: Topic results for each TopEx cluster obtained from the Sub-I Corpus of 665 student reflective writings submitted between August 2016 and July 2018 with example sentences and the number of sentences in each cluster. Cluster # (Theme #) # of Sentences Theme Description Example Sentence 0 (1) 131 Challenges around system issues. “As an AI [Sub-I], we are handicapped in that we can put in orders but that we need a cosigner even with oral approval.” 1 804 In-coherent cluster, no theme/challenge identified. N/A a 2 (2) 95 Difficulties adjusting to the style and preferences of different attendings. “The main challenge for me was adjusting to the numerous different attendings and evaluators on a day-to-day basis.” 3 1008 In-coherent cluster, no theme/challenge identified. N/A a 4 (4) 98 Developing a patient plan and differential diagnosis. “One of the biggest challenges I have faced thus far during my STICU rotation is having to develop a plan for complex patients in a very short amount of time.” 5 (3) 83 Increased sense of responsibility. “As many of the other students have mentioned, adapting to the additional responsibility is a big change for me.” 6 (9) 60 Patient and family-centered care. “The latter has been especially important as you try and keep families up to date, and where explaining complex procedures and medical care in such an acute setting can be difficult.” 7 (10) 91 Value of Night shifts. “I found that my week on nights with the hospitalist service was a wonderful educational opportunity because I felt more challenged like an intern than I do on my day shifts.” 8 (5) 101 Challenges of finding one’s role on a team. “I think one of the biggest challenges initially was learning what specific role I played on the team.” 9 (2) 112 Difficulties adjusting to the style and preferences of different attendings. “The transition from one team of residents to another was challenging because styles of handling patients differed.” 10 (6) 182 Challenges in the ambiguity of the learner. “I think the biggest challenge so far is being in this “in between zone” where we certainly experience an expanded role, responsibility, and independence as an M4 but not quite the full independence of an intern.” 11 (6) 104 Challenges in the ambiguity of the learner. “It was a difficult transition for me to have students that looked up to me to guide them in their presentations, differential diagnoses, and medical plans.” 12 70 In-coherent cluster, no theme/challenge identified. N/A a 13 (8) 25 Learners’ reflection on the pain they observed in the patients they cared for. “One particular challenge that I faced was taking care of patients coming in with sickle cell pain crises.” 14 (11) 73 Challenges specific to the ICU units “I have never rotated through an ICU before so learning about ventilators, vent weaning, lines/drips, etc has been challenging.” 15 (7) 117 Balancing one’s own learning with the workload and other students. “One challenge I have encountered, particularly in the initial weeks of the rotation has been learning how to be both a teacher to the M3 students under me while still learning myself.” 16 446 In-coherent cluster, no theme/challenge identified. N/A a 17 102 In-coherent cluster, no theme/challenge identified. N/A a 18 (5) 205 Challenges of finding one’s role on a team. “Overall the biggest struggle has just been trying to get a handle on things well enough to actually be a part of the team instead of just another medical student who needs to have their hand held.” 19 (1) 85 Challenges around system issues. “That said, once I got computer access, figuring out a new EMR and adjusting to a new team’s style of presentations, rounding, and a new hospital system are challenges that come up whenever I have to go to a new site.” Open in a new tab Abbreviations: Sub-I, Sub-Internship; AI, Acting Intern (aka Sub-Intern); ICU, Intensive Care Unit; STICU, Surgical Trauma Intensive Care Unit; M4, 4th Year Medical Student; M3, 3rd Year Medical Student; EMR, Electronic Medical Record a In-coherent cluster, no theme/challenge identified by the medical educators. See Appendix 4 for the TopEx-generated topics. Results We applied the TopEx pipeline to 665 student reflections (5,031 sentences) and obtained 20 clusters from 3,992 sentences with 1,021 sentences removed because they were too short (< 6 words) after sentence normalization. TopEx results were manually reviewed and interpreted by the authors to identify areas of possible improvement to the curriculum. Table 1 lists cluster themes defined by the authors and example sentences, Appendix 4 contains LDA topics identified by TopEx, and Appendix 5 is a visual representation of the sentence clusters. TopEx identified a range of challenges from technical issues with the electronic health record (EHR) to specific challenges of certain rotations to self-reflection on how to balance the work and one’s own learning to interacting with others on the medical team. Besides gaining insight into the challenges students faced transitioning into the Sub-I role with more clinical responsibilities, this data also revealed gaps in curriculum and structural design that could be improved. Specifically, theme 1 (clusters 0 and 19) includes the topic words “order”, “hospital” and “system” and include sentences that relay challenges around system issues such as entering orders and having a difficult time getting access to, or navigating, other hospital system EMRs during the internship. Theme 2 (clusters 2 and 9) has the topic words “attending”, “time”, “different”, “team”, and “resident” and include sentences that discuss difficulties adjusting to the style and preferences of different attendings. Cluster 5 comprises theme 3 and includes the topic words “responsibility” and “increased”, and is about the challenges related to the learners’ increased sense of responsibility for patients . Similarly, the fourth theme, cluster 4, is all about the challenges of coming up with a patient plan and differential diagnosis , and include the topic words “plan”, and “attending”. Theme 5 (clusters 8 and 18) include topic words “role”, “team”, “feel”, and “part”, and talk about the challenges of finding one’s role on a team . Clusters 10 and 11 create theme 6, which addresses the challenges in the ambiguity of the learner as a Sub-I, for example, navigating their role and the role of the student in order to balance their learning with the learning of the 3rd year medical students they were managing. This is reflected in the key topic words “intern”, “experience”, and “student.” Cluster 15 is theme 7 and includes sentences discussing how great of an opportunity the Sub-I rotation is, but that it has a steep learning curve, which is reflected by the topic words “learning”, “challenges”, “experience”, and “curve”. Topic words like “pain” and “chronic” are found in theme 8 (cluster 13), which is focused on the learners’ reflection of the pain they observed in the patients under their care. Other challenges were focused on a specific Sub-I rotation. For example, theme 9 (cluster 6) revolves around family and patient-centered care along with comments specific to family medicine with topic words such as “family”, “medicine”, “time” and “care”. Sentences in this cluster discuss challenges coordinating with specialists, the differences of family medicine versus other specialties, and the unique challenges involved in interacting with families. Theme 10 (cluster 7) is specifically focused on rotations for the night shift and the trauma rotation. This cluster includes topic words such as “night”, “trauma”, “team”, and “floor”, and identifies that it is a challenge to shift gears from working during the day to overnight; however, students found an abundance of learning opportunities during the night shift that were not available during the day shift. In addition, the trauma rotation included a lot of learning opportunities, but students found it challenging to keep up when the trauma floor became busy. In addition, theme 11 (cluster 14) is very specific with key topic words such as “ICU”, “challenge”, and “learning”. This cluster is focused on the specific challenges of the ICU rotation such as learning to use the ventilators and other devices unique to the intensive care setting. It was expected that not all clusters would be informative because some sentences were not related to challenges or not enough students discussed the topic. These clusters had no apparent theme to the authors, or had a theme that was not easily distinguished from other clusters (Clusters 1, 3, 12, 16, 17). The TopEx analysis of student reflections proved to be a useful formative tool to create a needs assessment for the Sub-I rotations. In response to some of the challenges identified by the analysis, Sub-I directors made a number of changes for the following academic year ( Table 2 ). Lastly, the findings regarding the family medicine Sub-I led to discussions with leaders of this Sub-I to address any issues unique to certain Sub-I practice locations. Table 2: Sub-I Curricular Changes Implemented with Associated Cluster Theme Data Cluster Theme (Number) Sub-I Actions Implemented Putting in Orders (#0) Incorporated order entry and admission order tips with examples into orientation for each rotation. Patient Care/Management (#4) Encouraged students to build in time during pre-rounds to research literature and guidelines to assist in management plans. Increased sense of responsibility (#5) During orientation, conveyed the expectation students assume responsibility for their patients as if they were an intern and seek help appropriately Patient and family-centered care (#6) Students are encouraged to participate in all communications with patients and their families as a part of their primary provider role during orientation expectation overview Night shifts (#7) We have encouraged all departmental Sub-I rotations to provide a night float (or similar) experience. Challenges specific to the ICU units (#14) Provided objectives for the ICU sub-internship that served as a roadmap for students to highlight concepts students should focus on learning. Ambiguity of the learner as a Sub-I (#10); Discomfort of transition (#11); Being a part of the team (#18) Emphasized roles and responsibilities during Sub-I orientation for each rotation, and are incorporating this into a traditional ward team. Non-NLP improvements b Added documentation orientation curriculum to ensure student notes met billing requirements as their notes became official medical records with attending attestation the following academic year. Open in a new tab Abbreviations: Sub-I, Sub-Internship; ICU, Intensive Care Unit b These are curriculum changes implemented that were not identified by the NLP analysis. Discussion Current course evaluation is done using generic end-of-course student assessments that often do not contain meaningful feedback and have poor response rates. This limits their usefulness as it may not be reflective of the students’ experience as a whole. Utilizing NLP, we were able to retrospectively mine medical student reflective writings and identify several aspects of the current medical curriculum that would benefit from changes to improve the student’s experience and learning. Additionally, this saved numerous hours of manual review of student reflections by the faculty Sub-I directors to elicit the same information. It is important to note that TopEx is not meant to be a one-stop-shop, automated curriculum evaluation tool. Rather it was designed to complement current evaluation workflows and does require human interpretation of the results. Specifically, TopEx is designed to identify common themes discussed by multiple students; thus, it will not identify topics discussed by a single student, which could include isolated security or safety incidents that should be captured by other reporting mechanisms. Users should also be aware that TopEx only utilizes part of each sentence for analysis, thus longer sentences that include multiple topics may not be properly represented. Additionally, short sentences are removed from the analysis, however, this threshold can be adjusted by the user. Finally, TopEx is an assistant and does not tell the user the answer. Instead it allows the researcher to quickly and efficiently explore the data by changing settings, which facilitates reflexivity. This allows the user to find the angle of the data that is the most helpful to their particular use case based on their experiences and knowledge of the area. As with all artificial intelligence methods, algorithmic bias is a concern as large machine learning models can be trained on vast amounts of public or private data that could influence the results. With TopEx, the mitigation of bias is directly under the control of the user as the numerical sentence representations are generated directly from the user’s input corpus. For instance, if the cohort being analyzed only consists of students of high economic status or specific demographics, then the results will be biased towards challenges this cohort of students is experiencing and will miss challenges students from underrepresented socioeconomic or demographic cohorts experience. With TopEx, this bias can be mitigated by having a good understanding of the input dataset. Ensuring the data being input into the system is balanced and equitable will help mitigate bias in the results. However, this cannot always be controlled as one may not have control over data that is being analyzed retrospectively, but it can be considered downstream during the results interpretation stage. While actions were taken to improve the Sub-I experience for our students based on the NLP analysis, this did not capture all course feedback received nor foreshadow improvements in the curriculum that were needed ( Table 2 ). This was due to the nature of the blog post prompts, which were not designed for, nor asked for, specific course feedback. Additionally, professionalism concerns are gathered with course and faculty feedback, which are not captured, nor appropriate in this dataset. Based on the identified themes and challenges faced by students on their Sub-I rotation, directors made a number of improvements in the curriculum that had not been previously identified on course evaluations. These are highlighted in Table 2 and included incorporating curriculum into our orientation day half-day session on placing orders and how to approach admission orders in the electronic medical record, encouraging departmental Sub-I directors to offer night float experience during the Sub-I rotation as a general expectation, and provided specific ICU objectives that highlight concepts and skills students should focus on learning and achieving during their rotation. Next steps involve fine-tuning the reflective writing prompts to provide more cohesive responses that will aid in challenge identification by TopEx, and obtaining student feedback to ensure topics are congruent with their posts. Expanding the use of TopEx to be used at regular intervals, applying to narrative courses and/or faculty feedback, and utilizing this tool to evaluate other medical curricula will aid medical educators in quickly identifying actionable items from qualitative data to improve their medical education programs. In addition, while TopEx has only been formally evaluated on the Sub-I Corpus ( Olex et al. 2020 ), it has been found useful in analyzing other types of medical education corpora such as clinical skills self-assessment narratives ( Turner et al, 2024 ) and a variety of other text data ( Olex et al., 2021 ) for uses beyond curriculum evaluation. Thus, TopEx could aid in analyzing large swaths of narrative information that would otherwise be time-intensive to process manually. Conclusion NLP improved our ability to assess student experiences during rotations, and emerged as a useful tool to integrate, previously unused, student reflections that were not targeted towards curriculum feedback into the curriculum evaluation process. The advantages of using NLP in curriculum evaluation are two-fold. First, NLP provides an avenue for the secondary use of narrative texts written from the student perspective into curriculum evaluation that would have not been considered otherwise. Second, the use of this methodology provides a robust and time efficient analysis of a large corpus of data for clinician educator interpretation and review. Supplementary Material Supplementary Appendices NIHMS2015653-supplement-Supplementary_Appendices.docx (822KB, docx) Acknowledgements The authors would like to thank John W. Cyrus, Research and Educational Librarian, for his collaboration in providing a literature review for this work. The authors would also like to thank Dr. Bridget T. McInnes, Associate Professor of the Department of Computer Science for consulting on the development of TopEx and a critical review of this manuscript. Funding/Support This work was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through grant UL1TR002649. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health. Biographies AL Olex is Senior Scientist at Virginia Commonwealth University Wright Regional Center for Clinical and Translational Science, Richmond, Virginia. A Garber is Associate Professor, Internal Medicine at Virginia Commonwealth University School of Medicine, Richmond, Virginia, and VCU School of Medicine Acting Internship Director. C Blondino was a Research Assistant/PhD candidate at Virginia Commonwealth University School of Medicine, Richmond, Virginia and is now an Assistant Professor of Health Sciences at the University of Richmond. SA Santen is senior associate dean, assessment, evaluation and scholarship and professor of Emergency Medicine at Virginia Commonwealth University School of Medicine, Richmond, Virginia and professor of Emergency Medicine and Medical Education at University of Cincinnati College of Medicine. S Goldberg was Associate Professor, Surgery at Virginia Commonwealth University School of Medicine, Richmond, Virginia and is now at Mary Washington Healthcare, General Surgery Department. D DiazGranados is Associate Professor, Psychiatry at Virginia Commonwealth University School of Medicine and Director of Evaluation for the Wright Regional Center for Clinical and Translational Science, Richmond, Virginia. Footnotes Disclosure Statement The authors report there are no competing interests to declare. References Blei David M., Ng Andrew Y., and Jordan Michael I.. 2003. “Latent Dirichlet Allocation.” The Journal of Machine Learning Research 3: 993–1022. [ Google Scholar ] Chen Yukun, Wrenn Jesse, Xu Hua, Spickard Anderson 3rd, Habermann Ralf, Powers James, and Denny Joshua C.. 2014. “Automated Assessment of Medical Students’ Clinical Exposures according to AAMC Geriatric Competencies.” AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium 2014 (November): 375–84. [ PMC free article ] [ PubMed ] [ Google Scholar ] Denny Joshua C., Spickard Anderson 3rd, Speltz Peter J., Porier Renee, Rosenstiel Donna E., and Powers James S.. 2015. “Using Natural Language Processing to Provide Personalized Learning Opportunities from Trainee Clinical Notes.” Journal of Biomedical Informatics 56 (August): 292–99. [ DOI ] [ PubMed ] [ Google Scholar ] Dias Roger D., Gupta Avni, and Yule Steven J.. 2019. “Using Machine Learning to Assess Physician Competence: A Systematic Review.” Academic Medicine: Journal of the Association of American Medical Colleges 94 (3): 427–39. [ DOI ] [ PubMed ] [ Google Scholar ] Mann Karen, Gordon Jill, and Anna MacLeod. 2009. “Reflection and Reflective Practice in Health Professions Education: A Systematic Review.” Advances in Health Sciences Education: Theory and Practice 14 (4): 595–621. [ DOI ] [ PubMed ] [ Google Scholar ] Olex Amy L., Deborah DiazGranados, McInnes Bridget T., and Goldberg Stephanie. 2020. “Local Topic Mining for Reflective Medical Writing.” AMIA Summits on Translational Science Proceedings 2020 (May): 459–68. [ PMC free article ] [ PubMed ] [ Google Scholar ] Olex Amy L., French Evan, Burdette Peter, Sagiraju Srilakshmi, Neumann Thomas, Gal Tamas S., and McInnes Bridget T.. 2021. “TopEx: Topic Exploration of COVID-19 Corpora - Results from the BioCreative VII Challenge Track 4”. In Proceedings of the BioCreative VII Challenge Evaluation Workshop, (Virtual), pp. 238–242. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Spickard Anderson 3rd, Ridinger Heather, Wrenn Jesse, Nathan O’brien, Adam Shpigel, Wolf Michael, Stein Glenn, and Denny Joshua. 2014. “Automatic Scoring of Medical Students’ Clinical Notes to Monitor Learning in the Workplace.” Medical Teacher 36 (1): 68–72. [ DOI ] [ PubMed ] [ Google Scholar ] Turner L, Weber DE, Santen SA, Olex AL, Baker P, Overla S, Shu D, Randolph A, Kelleher M, 2024. Making Use of Natural Language Processing to Better Understand Medical Students’ Self-Assessment of Clinical Skills. Academic Medicine 99, 285. 10.1097/ACM.0000000000005527 [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Winkel Abigail Ford, Yingling Sandra, Jones Aubrie-Ann, and Nicholson Joey. 2017. “Reflection as a Learning Tool in Graduate Medical Education: A Systematic Review.” Journal of Graduate Medical Education 9 (4): 430–39. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Supplementary Materials Supplementary Appendices NIHMS2015653-supplement-Supplementary_Appendices.docx (822KB, docx) ACTIONS View on publisher site PDF (300.9 KB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• Harnessing ChatGPT for Medical Education: A Narrative Review.#223037
• AI in the Classroom: Observing Preclinical Students' Use of ChatGPT During Case‐Based Learning at a UK Medical School#1053080
• A 55-Word Story Narrative Medicine Workshop to Promote Reflective Practice Among Internal Medicine Residents.#1134514
• Improving patient communication and understanding of medical reports through text simplification with ChatGPT: a comparative study.#1107385
• The Role of Large Language Models in Medical Education: Applications and Implications#976479