Evaluating AI-Assisted L2 Writing Revision through a Multi-Objective Framework: Effectiveness, Style Alignment, and Teacher-Referenced Benchmarking

This study evaluates the effectiveness of generative artificial intelligence (AI) in second language (L2) writing revision through a multi-objective framework that integrates improvement, style alignment, and teacher-referenced benchmarking. Although prior research on AI-assisted writing has primarily emphasized outcome gains, considerably less attention has been paid to whether AI-generated revisions preserve the original meaning and discourse structure of learner texts, or how such revisions align with pedagogically grounded standards. To address this gap, the present study employed a within-item repeated-measures design in which 120 student essays were independently revised by four large language models (Doubao-Seed-1.6, DeepSeek-V3, Gemini-2.5-pro, and GPT-5) as well as by an experienced teacher. Revision quality was assessed from two complementary dimensions: score improvement and style alignment, with the latter reflecting the extent to which revisions preserved the original meaning while avoiding excessive rewriting. The findings revealed substantial variation across models depending on the evaluation criterion adopted. Under gain-oriented evaluation, Gemini-2.5-pro and Doubao-Seed-1.6 achieved the greatest score improvements. In contrast, teacher revisions consistently demonstrated the strongest alignment with the original texts, whereas several AI systems tended to introduce more extensive modifications. When multiple evaluation objectives were jointly considered, Gemini-2.5-pro most consistently achieved a balanced performance across criteria, while Doubao-Seed-1.6 was selected less frequently due to weaker alignment performance. These findings suggest that score improvement alone is insufficient for evaluating AI-assisted writing revision in educational settings. More broadly, the study underscores the importance of balancing improvement, meaning preservation, and revision effort, and proposes a teacher-referenced evaluative framework for the responsible integration of generative AI feedback into L2 writing instruction.