Theory-Based and ITS-Integrated AI Feedback Versus Teacher Feedback on Reflective Writing in K–12 Education: A Comparative Analysis of Content Structure and Learner Receptivity Based on the Four Levels of Feedback
TL;DR
Findings indicate that ITS-integrated AI feedback offers an efficient, theoretically grounded complement to teacher feedback in K-12 education, focusing on feedback characteristics and learner receptivity.
Abstract
This research compares AI-generated and teacher feedback on student reflections in Japanese K-12 classrooms that adopt self-paced learning, focusing on feedback characteristics and learner receptivity. Two AI conditions were designed based on Hattie and Timperley’s four-level feedback model: Theory-AI, employing GPT-4.1-mini with this framework alone, and Context-AI, employing GPT-5.1 with an Intelligent Tutoring System (ITS) architecture that integrates learner and domain models. These were compared with teacher feedback (Teacher) in a within-subject design involving 110 students across four schools and five grade levels. Feedback characteristics (RQ1) were analyzed along three dimensions—volume, generation time, and content composition across the four levels (task, process, self-regulation, and self)—and their uniformity both between and within class groups. Learner receptivity (RQ2) was assessed via clarity, specificity, and empathy on a 5-point scale. For RQ1, both AI conditions exhibited higher proportions of process- and self-regulation-level feedback than teachers, whereas Theory-AI produced excessive self-level feedback. AI feedback was generated approximately ten times faster and exhibited substantially greater uniformity in both volume and content composition, whereas teacher feedback varied considerably between class groups (e.g., self-level inclusion: Cramér’s $V = .468$ ) and within the same class group (character-count SD: 9.4–37.1). For RQ2, no differences were observed in clarity; specificity ratings followed Context-AI > Theory-AI > Teacher, and empathy ratings followed Theory-AI > Context-AI > Teacher, with all pairwise differences significant. These findings indicate that ITS-integrated AI feedback offers an efficient, theoretically grounded complement to teacher feedback in K-12 education.