Distinguishing performance gains from learning when using generative AI Lixiang Yan1*, Samuel Greiff2 , Jason M. Lodge3 , Dragan Gašević1 1* Faculty of Information Technology, Monash University, Clayton,
arXiv:2605.13731v1 [cs.LG] 13 May 2026
Victoria, Australia. 2 School of Social Sciences and Technology, Technical University of
Munich, Munich, Germany. 3 School of Education, The University of Queensland, Brisbane,
Queensland, Australia.
*Corresponding author(s). E-mail(s): [email protected];
Accepted manuscript / postprint version This is the accepted manuscript version of the following article: Yan, L., Greiff, S., Lodge, J. M., & Gašević, D. (2025). Distinguishing performance gains from learning when using generative AI. Nature Reviews Psychology, 4, 435–436. https://doi.org/10.1038/s44159-025-00467-5 The final authenticated version is available online at: https://www.nature.com/articles/s44159-025-00467-5 © Springer Nature Limited 2025. This accepted manuscript is made available in accordance with the Springer Nature self-archiving policy. This manuscript may not be redistributed or reused for commercial purposes and may not be released under a Creative Commons licence.
Standfirst Generative artificial intelligence (AI) is increasingly being integrated into education, where it can boost learners’ performance. However, these uses do not promote the deep cognitive and metacognitive processing that are required for high-quality learning. 1
Main The rapid adoption of generative artificial intelligence (AI) tools such as ChatGPT is transforming how learners engage with information, especially in tertiary education. Celebrated for providing instant assistance and personalized feedback, these tools promise to enhance performance across tasks such as academic writing, computer programming and information research[1, 2]. Benefits such as improved task performance and reduced cognitive load have been reported for tertiary students working on writing and information search tasks[3]. However, beneath these immediate gains lies a paradox: although generative AI can boost learners’ performance on tasks, it risks undermining the cognitive and metacognitive processes that are essential for durable learning[3, 4]. There is a fundamental distinction between performance and learning[5]. Performance refers to observable behaviours during task execution, often influenced by external factors or supports. By contrast, learning involves enduring changes in knowledge or behaviour that result from experience; learning is characterized by the ability to independently retain and transfer skills[5]. It is well established that high performance does not always imply learning, as performance metrics are often transient and contextually dependent[5]. For example, a student might achieve high performance on a test after massed practice (‘cramming’) the night before, yet fail to retain or apply the knowledge later, indicating performance without lasting learning[5]. This fundamental distinction is underexamined in emerging research on generative AI in education. For instance, a meta-analysis of 69 studies aimed to investigate the impact of ChatGPT and other generative AI tools on “student learning” in K-12 and tertiary education. Yet, the meta-analysis reported that these technologies improve student “academic performance”, with an effect of (Hedge’s) g = 0.7, reflecting the pervasive conflation of these concepts[1]. Although the reported effect size is large, it probably reflects immediate task success rather than learning. Another study identified significant boosts to undergraduate student performance in providing quality peer feedback when using generative AI-based assistance[6]. However, these performance improvements diminished once the generative AI assistance was removed, indicating that immediate gains in performance might not reflect sustained improvements in student learning.
Cognitive load, metacognition and self-efficacy Clarifying the distinction between performance and learning requires examination of how generative AI influences the key psychological processes involved in durable learning. Specifically, balancing cognitive load, metacognition and self-efficacy are critical to achieving deeper, lasting learning[5, 7, 8]. Investigating these constructs can help to reveal whether immediate performance gains from generative AI translate into sustained improvements in knowledge and skills. Generative AI tools can in some cases reduce cognitive load, particularly for tasks that require complex information processing. For instance, students who used ChatGPT to research socio-scientific issues reported significantly lower cognitive load compared with those who used more traditional research tools such as internet search 2
engines[3]. From the perspective of cognitive load theory[7], offloading domain-specific tasks to generative AI can free up learners’ working memory resources. However, reduced cognitive load does not necessarily lead to deeper cognitive engagement with the learning material. Indeed, excessively offloading cognitive tasks to generative AI risks reducing active cognitive engagement, which is crucial for deep encoding, effective retention and meaningful transfer. This risk is supported by evidence showing that undergraduate students who relied heavily on ChatGPT for information gathering exhibited weaker argumentation and reasoning skills compared with those who used traditional, cognitively demanding research methods[3]. Generative AI might also disrupt metacognition. A randomized controlled study indicated that frequent use of generative AI for argumentative essay writing tasks can foster “metacognitive laziness” in undergraduate students[4]. That is, learners offloaded critical evaluative tasks, such as reviewing task criteria, assessing written drafts against rubrics and reflecting critically on learning materials, to the tool rather than actively reflecting on their work. These evaluative tasks are crucial because they enable learners to independently judge the quality of their work, identify areas that need improvement and actively regulate their own learning process. This offloading undermines key self-regulation skills by reducing learners’ active involvement in these metacognitive activities. Psychological theories of over-reliance suggest that external tools can displace internal cognitive effort, reducing learners’ ability to independently retain and transfer knowledge[9]. Offloading metacognitive activities to generative AI can also stunt intrinsic motivation, as the learner’s role in task completion diminishes[4]. From the perspective of self-determination theory, intrinsic motivation depends on learners’ autonomy and active engagement[8]. Delegating key metacognitive tasks, such as planning, monitoring and evaluating, to generative AI might reduce learners’ active participation and sense of control, thereby decreasing intrinsic motivation. Along these lines, undergraduate learners who interacted frequently with generative-AI-powered chatbots for acquiring academic domain-specific knowledge reported lower learning autonomy relative to students who had infrequent chatbot interactions[9]. Generative AI’s paradoxical effects on self-efficacy further complicate its role in human learning. For instance, a study reported that frequent generative AI use among university students enhanced learners’ perceived confidence and efficiency during academic tasks but simultaneously increased technological dependence[10]. This technological dependence can have long-term repercussions, eroding learners’ ability to engage in independent learning and reducing their resilience in the face of new challenges[5].
Targeting learning in research and practice Harnessing the potential of generative AI requires us to rethink research design and interpretation. To address the conceptual and methodological limitations identified above, we propose a research agenda that is informed by cognitive psychology. Clarifying how generative AI tools influence individuals’ encoding, consolidation and retrieval processes is essential for determining whether these technologies enable learning or 3
merely confer short-term performance advantages. By illuminating the underlying cognitive mechanisms, researchers can better evaluate to what extent generative AI-based educational assistance promotes deep engagement, transfer of knowledge and durable skill development, rather than simply facilitating task completion. First and foremost, researchers should seek to differentiate learning from performance. To do so, researchers should use process-oriented assessments such as retention and transfer tests to distinguish genuine learning from mere task performance. Examination of testing effects and use of delayed recall tasks can be used to evaluate durable learning in AI-supported contexts[5]. Second, researchers should investigate how generative AI influences encoding, consolidation and retrieval processes. For instance, the relationship between cognitive load and deep encoding remains to be clarified. Paradigms from cognitive load theory can be used to explore these effects. Finally, and in conjunction with the recommendations above, researchers should study the longitudinal effects of educational generative AI tools. The long-term effects of AI tools on knowledge retention and skill development and the interaction between AI-assisted learning and prior knowledge remain to be examined. Educators and designers of educational generative AI tools can also take steps to ensure that generative AI-supported tasks promote learning rather than just performance gains. First, they can promote active metacognition by encouraging learners to critically evaluate AI-generated outputs rather than accepting them passively. Second, educators and practitioners should balance efficiency with autonomy. Generative AI should be used to support, not replace, independent cognitive effort. Furthermore, feedback from generative AI chatbots can be blended with opportunities for unaided problem-solving to foster students’ autonomy alongside task success.
Outlook Although generative AI tools such as ChatGPT can enhance learners’ performance and reduce cognitive load, they might disrupt the deeper processes of encoding, retention and independent problem-solving that are necessary for durable learning. By prioritizing process-based assessments, clarifying how generative AI interacts with cognitive functions and examining long-term effects, researchers can develop a more nuanced understanding of human–AI interactions. Ultimately, generative AI must serve as an augmentation for human learning, not a replacement, and we must ensure that learners develop knowledge and skills that extend beyond the confines of generative AI-assisted tasks.
Competing interests The authors declare no competing interests.
References [1] Deng, R., Jiang, M., Yu, X., Lu, Y. & Liu, S. Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Computers & Education 227, 105224 (2025). 4
[2] Yan, L., Greiff, S., Teuber, Z. & Gašević, D. Promises and challenges of generative artificial intelligence for human learning. Nature Human Behaviour 8, 1839–1850 (2024). [3] Stadler, M., Bannert, M. & Sailer, M. Cognitive ease at a cost: LLMs reduce mental effort but compromise depth in student scientific inquiry. Computers in Human Behavior 160, 108386 (2024). [4] Fan, Y. et al. Beware of metacognitive laziness: effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology 56, 489–530 (2024). [5] Soderstrom, N. C. & Bjork, R. A. Learning versus performance: an integrative review. Perspectives on Psychological Science 10, 176–199 (2015). [6] Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D. & Siemens, G. Impact of AI assistance on student agency. Computers & Education 210, 104967 (2024). [7] Sweller, J. Cognitive load theory. In Mestre, J. P. & Ross, B. H. (eds) Psychology of Learning and Motivation, Vol. 55, 37–76 (Academic, 2011). [8] Ryan, R. M. & Deci, E. L. Intrinsic and extrinsic motivation from a selfdetermination theory perspective: definitions, theory, practices, and future directions. Contemporary Educational Psychology 61, 101860 (2020). [9] Zhai, C., Wibowo, S. & Li, L. D. The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: a systematic review. Smart Learning Environments 11, 28 (2024). [10] Zhang, L. & Xu, J. The paradox of self-efficacy and technological dependence: unraveling generative AI’s impact on university students’ task completion. The Internet and Higher Education 65, 100978 (2025).
5