Generative AI in clinical practice: novel qualitative evidence of risk and responsible use of Google's NotebookLM.

Generative AI in clinical practice: novel qualitative evidence of risk and responsible use of Google’s NotebookLM - PMC Skip to main content An official website of the United States government Here's how you know Here's how you know Search Log in Primary site navigation Search Log in Search PMC Full-Text Archive Search in PMC Journal List User Guide As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsement of, or agreement with, the contents by NLM or the National Institutes of Health. Learn more: PMC Disclaimer | PMC Copyright Notice letter Eye (Lond) . 2025 May 2;39(8):1650–1652. doi: 10.1038/s41433-025-03817-y Generative AI in clinical practice: novel qualitative evidence of risk and responsible use of Google’s NotebookLM Max Reuter 1 , Maura Philippone 2 , Bond Benton 3 , Laura Dilley 2, ✉ Author information Article notes Copyright and License information 1 Department of Computer Science & Engineering, Michigan State University, East Lansing, MI USA 2 Department of Communicative Sciences & Disorders, Michigan State University, East Lansing, MI USA 3 School of Communication and Media, Montclair State University, Montclair, NJ USA ✉ Corresponding author. Received 2025 Mar 22; Revised 2025 Apr 6; Accepted 2025 Apr 24; Issue date 2025 Jun. Subject terms: Education, Health occupations © The Author(s), under exclusive licence to The Royal College of Ophthalmologists 2025 PMC Copyright notice PMCID: PMC12089454  PMID: 40316783 The advent of generative artificial intelligence, especially large language models (LLMs), presents opportunities for innovation in research, clinical practice, and education. Recently, Dihan et al. [ 1 ] lauded LLM tool NotebookLM’s potential, including for generating AI-voiced podcasts to educate patients about treatment and rehabilitation, and for quickly synthesizing medical literature for professionals. We argue that NotebookLM presently poses clinical and technological risks that should be tested and considered prior to its implementation in clinical practice. Figure 1 presents examples of NotebookLM’s shortcomings in fact-checking and reliability. As our testing demonstrates, when provided with inaccurate sources, NotebookLM may generate outputs unsupported by research (e.g., advising providers to tell their patients to eat rocks, as shown in the top-left portion of Fig. 1 ). Moreover, even with access to accurate information, NotebookLM may still generate inaccurate outputs (e.g., claiming that there are 16 sources in total when there are actually 20, as shown in the bottom-right portion of Fig. 1 ). These examples contrast claims by Dihan et al. [ 1 ] that NotebookLM may presently be used to optimize medical education and service provision. Fig. 1. Inaccurate responses given by NotebookLM to user queries; output is stylized for visual clarity. Open in a new tab NotebookLM is accessed in a web browser, where users can upload up to 50 documents to serve as the AI’s knowledge base. Users can type questions about the documents to a chatbot, whose responses include hyperlinked numbers that highlight portions of the documents the chatbot used while formulating its reply. Users can generate an audio-only podcast wherein a perceptually male and female host distill the documents into an informal discussion. Top-left: NotebookLM advises the user to tell their patients that eating rocks is healthy, citing the user’s document. Top-right: NotebookLM tells the user 2 + 2 = 5, citing the user’s document. Bottom-left: Given an input of 20 documents on the topic of pediatric voice disorders, NotebookLM fails to list documents whose data or viewpoints conflict with each other, instead citing document 1 repeatedly (red circles). Bottom-right: NotebookLM fails to output the correct number of sources uploaded by the user. Further, Table 1 outlines passages in ref. [ 1 ] that raise concerns about patient data protection, generated summaries of user documents, patient education, factual accuracy, and citation reliability. Using examples from both existing literature and our testing, Table 1 identifies various clinical, ethical, and reliability concerns related to the authors’ claims in these areas. Table 1. Passages from Dihan et al. advocating for use of NotebookLM (Column 1) which are associated with clinical and/or ethical concerns (Column 2), or else accuracy and/or reliability concerns (Column 3). Dihan et. al passage Clinical or ethical concerns Accuracy or reliability concerns Patient data protection “Though NotebookLM is a commercial entity that does not abide by patient privacy regulations, it does represent an archetype for the future potential of source- grounded LLM integration into patient electronic medical record systems. […] [Future source-grounded LLMs] may have valuable implications in being able to quickly access and synthesize information from a patient’s entire medical record to answer pertinent questions, potentially saving time associated with lengthy chart reviews.” Documents uploaded by free-tier users may be subject to human review or used to train other AI at Google’s discretion [ 1 ], potentially jeopardizing patient data protection, privacy, and both patients’ and providers’ consent to data repurposing [ 2 ]. Even HIPAA may be insufficient to address such privacy risks, necessitating cautious use and responsible development to protect patient data [ 3 ]. LLMs can overlook or misinterpret key information in any set of information, especially when tasked with summarizing a large amount of content, potentially resulting in misguided medical decisions. Lit. summs. “A podcast generator can improve the way ophthalmologists, trainees, and medical students manage their time by summarizing primary ophthalmology sources and complex research papers into easily digestible audio content.” Summaries of uploaded materials may contain outdated or discredited information if a source contains such content (see Fig. 1 for examples), potentially misleading patients and medical professionals. Given any set of documents, and especially those containing complex documents, LLMs may misinterpret and subsequently misrepresent some of their contents. Patient education “Rather than requiring active visual engagement through reading, podcasts allow listeners to absorb information while performing other tasks, allowing for a more flexible form of learning.” Media multitasking would be ill-advised given a demonstrated risk of negatively impacting learning outcomes, with research evidencing reduced implicit learning and/or possibly reduced daily task performance with greater magnitude of multitasking activities in some populations [ 4 , 5 ]. Generated podcasts may over-simplify or misrepresent the source content in order to provide a more informal conversation, and may use outdated terminology. Factuality “[NotebookLM] learns from user-uploaded multimodal information—such as documents, images, and videos—to summarize and generate accurate, contextually grounded outputs about them.” NotebookLM can neither identify misinformation contained within uploaded files nor incorporate relevant information beyond the uploaded content. Despite Google’s intent for NotebookLM to ground its outputs in the source material, it may still erroneously output potentially misleading content not included in the source material. Citations “[NotebookLM’s] citations are automatically generated for all content that NotebookLM pulls from within these materials, streamlining the fact-checking process.” The fact-checking process may be impossible or error-prone due to confusion caused by un-clickable or hallucinated citations. Hallucinations may lead to unexpected issues in the software, causing the user interface to become unusable or behave in unintended ways. Open in a new tab Rows in the table highlight categories of concern with their framing, which relate to: patient data protection (Row 1), summaries of literature provided by NotebookLM (Row 2), use of NotebookLM for patient education (Row 3), factuality of output (Row 4), and abilities to verify citations (Row 5). Importantly, using NotebookLM to educate medical professionals presently risks misleading them, as NotebookLM’s lack of fact-checking allows it to propagate outdated or discredited information. Additionally, since LLMs are prone to erroneously fabricating information (known as hallucinating ), the fact-checking process may be prone to error if some of NotebookLM’s citations are hallucinated, rendering them invalid. In conclusion, while NotebookLM presents intriguing potential for innovating on medical and clinical research and education, known and previously unreported limitations in NotebookLM warrant careful consideration before it can be deployed responsibly in clinical practice. Current risks include attenuated treatment outcomes, jeopardized patient privacy, and the propagation of false or misleading information to patients and medical professionals. Author contributions MR: drafting, editing, reviewing, methodology; MP: drafting, editing, reviewing, methodology; BB: drafting, reviewing, editing; LD: conceptualization, drafting, editing, reviewing, methodology, and approval of the final version. Data availability The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request. 1 NotebookLM’s data policy as of 6 April 2025 is available at http://web.archive.org/web/20250406034254/https://support.google.com/notebooklm/answer/15724963 . Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. References 1. Dihan QA, Nihalani BR, Tooley AA, Elhusseiny AM. Eyes on Google’s NotebookLM: using generative AI to create ophthalmology podcasts with a single click. Eye. 2025;39:215–6. [ DOI ] [ PMC free article ] [ PubMed ] 2. Banja J. How might artificial intelligence applications impact risk management?. AMA J Ethics. 2020;22:945–51. [ DOI ] [ PubMed ] [ Google Scholar ] 3. Marks M, Haupt CE. AI chatbots, health privacy, and challenges to HIPAA compliance. JAMA. 2023;330:309–10. [ DOI ] [ PubMed ] [ Google Scholar ] 4. Edwards KS, Shin M. Media multitasking and implicit learning. Attention Percept Psychophys. 2017;79:1535–49. [ DOI ] [ PubMed ] [ Google Scholar ] 5. Alexandru Florian C, Camelia Corina P, Adelina M, Vlad C, Roxana Ramona O, Emil Robert S, et al. Dual-task performance and balance in patients with severe COPD: a cross-sectional study. Therapeutic Adv Respir Dis. 2024;18:17534666241287302. [ DOI ] [ PMC free article ] [ PubMed ] [ Google Scholar ] Associated Data This section collects any data citations, data availability statements, or supplementary materials included in this article. Data Availability Statement The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request. 1 NotebookLM’s data policy as of 6 April 2025 is available at http://web.archive.org/web/20250406034254/https://support.google.com/notebooklm/answer/15724963 . Articles from Eye are provided here courtesy of Nature Publishing Group ACTIONS View on publisher site PDF (2.3 MB) Cite Collections Permalink RESOURCES Similar articles Cited by other articles Links to NCBI Databases Follow NCBI NCBI on X (formerly known as Twitter) NCBI on Facebook NCBI on LinkedIn NCBI on GitHub NCBI RSS feed Connect with NLM NLM on X (formerly known as Twitter) NLM on Facebook NLM on YouTube National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894 Web Policies FOIA HHS Vulnerability Disclosure Help Accessibility Careers NLM NIH HHS USA.gov Back to Top

Related documents

• Generative Artificial Intelligence in Healthcare: Automation Bias, Deskilling, and Cognitive Implications - A Systematic Review.#367919
• Risks and benefits associated with the development of generative AI technology.#69021
• A generative AI-enhanced intelligent service system with contextual retrieval and adaptive interaction for hospital use.#440682
• The imperative for regulatory oversight of large language models (or generative AI) in healthcare#328699
• A Control-Based Model of Authorship in AI-Mediated Research#127015