Integrating LLMs and Academic Writing Instruction: Engaged Reading and LLM-Generated Material

Integrating Large Language Models (LLMs) into course designs in the higher education context remains an open and quickly growing area of research. This study contributes to the emerging literature by examining how students engage with and evaluate LLM-generated feedback within academic writing instruction. Building on prior work on integration and LLM detection, the study adopts a diachronic design to assess whether students can reliably distinguish between human- and LLM-generated reviews, and how engagement with material may affect perceived classification. Fifty-one undergraduate participants received both human and LLM-generated reviews for essays they authored, completing surveys before and after revising their work. The results indicate that students were initially at random odds in distinguishing human from LLM-generated texts but exhibited statistically significant improvement following engagement. Nearly all students reported positive attitudes toward personalized feedback, irrespective of whether they perceived it to be human or LLM-generated. The findings underscore both the pedagogical potential and evaluative ambiguity of LLM integration in writing instruction.