Human-centered GenAI feedback design in higher education: a multisite experiment on direct, reflective, and hybrid approaches to scientific argumentation

Abstract

Generative artificial intelligence (GenAI) is increasingly used for feedback in higher education, yet evidence remains limited on how alternative human–AI feedback designs shape learning processes and durable outcomes. This study addresses that gap through a multisite, cluster-randomized, longitudinal field experiment comparing four feedback designs in introductory university science courses: peer feedback only, direct GenAI-supported feedback, reflective GenAI-supported feedback, and a hybrid design combining self-evaluation, peer feedback, and GenAI critique. The analytic sample comprised 1,176 first-year undergraduate students from 48 course sections across four universities and three science domains. Primary and secondary outcomes were argument-quality gain on four shared rubric dimensions—claim quality, evidence relevance and sufficiency, coherence of reasoning, and treatment of limitations or alternative explanations—conceptual learning, and delayed AI-free transfer; feedback uptake and self-regulated learning during revision were modeled as process mediators. Direct GenAI-supported feedback improved immediate argument-quality gain relative to peer feedback, whereas reflective and hybrid designs produced stronger feedback uptake and self-regulated learning. The hybrid condition yielded the highest adjusted mean for immediate argument-quality gain and showed the clearest advantage on conceptual learning; the reflective condition showed a positive but non-significant adjusted contrast on conceptual learning relative to direct GenAI-supported feedback. Both reflective and hybrid conditions outperformed direct GenAI-supported feedback on delayed AI-free transfer. Multilevel mediation analyses indicated that feedback uptake and self-regulated learning partially explained these advantages. By comparing four feedback designs, modeling revision processes, and assessing delayed AI-free transfer in a multisite field experiment, the findings suggest that the educational value of GenAI in higher education may depend less on AI access per se than on whether feedback environments preserve student agency, evaluative judgment, and ownership during revision.