Long-term impact of ChatGPT feedback on EFL students' writing skills development accross genres

Abstract

This study examined the long-term effect of ChatGPT feedback on EFL students' writing skills development across multiple genres. While Automated Writing Evaluation (AWE) tools have proven effective in improving surface level accuracy, few studies have assessed their sustained influence on higher-order writing skills, such as developing content and organizing ideas. In particular, empirical evidence on how AI-generated feedback supports learners over an extended instructional period and across different rhetorical contexts remains scarce. This research employed a quasiexperimental time-series design. The study was conducted over one semester (16 meetings), involving 36 EFL students enrolled in paragraph writing course. The students engaged in iterative writing and revision cycles, using ChatGPT feedback supplemented by teacher guidance that targeted both higher-level and lower-level writing skills. Writing samples covering comparison contrast, cause‒effect, and argumentative genres were analyzed through repeated-measures ANOVA to determine changes over time and across text types. The results revealed a significant improvement in students’ writing scores across all genres, F(2, 68) = 8.365, p = .001, partial η² = .197, with the greatest gains observed in argumentative writing. The relationship between genre and time was also significant, indicating genre-specific learning patterns. Notably, improvement in students’ initial drafts across subsequent writing tasks suggests that learners were able to apply previously addressed aspects of writing with increasing independence. This indicates that internalization of feedback and transfer of learning across genres emerged. These findings further demonstrate that students’ writing development is not uniform but influenced by task complexity and prior learning experiences. The findings suggest that ChatGPT can serve as a valuable mediational tool when integrated with teacher scaffolding, fostering sustainable development of both higher and lower-level writing skills. The study underscores the potential of AI in EFL context and calls for informed teacher mediation to maximize learning benefits and mitigate overreliance on automated feedback.