Enhancing PISA-aligned argumentation skills of prospective biology teachers through generative artificial intelligence

Abstract

Despite scientific argumentation being a fundamental competency of the PISA assessment framework, prospective biology teachers often demonstrate insufficient development in evidence-based reasoning and rebuttal construction skills. Considering the limitations of large-scale implementation of individualized approaches in higher education, this study evaluates the effectiveness of Generative Artificial Intelligence (ChatGPT-4) configured as a “Socratic tutor” in developing argumentation skills among future educators. An 8-week quasi-experimental study involving 124 students from a pedagogical university employed traditional peer discussion methods in the control group ( n = 62), while the experimental group ( n = 62) utilized an AI agent that induced cognitive dissonance regarding socio-scientific issues. Analysis of Covariance (ANCOVA) of data obtained through Toulmin's Argument Pattern (TAP) and PISA rubrics confirmed that the experimental group achieved statistically significantly higher overall argumentation quality compared to the control group (F (1,121) = 14.23, p < .001, η 2 = .105). Particularly, participants who received AI-assisted instruction showed marked improvement in forming rebuttals and evaluating counterevidence, which are considered among the most cognitively demanding components within the PISA scientific literacy framework (OECD). These findings suggest that a Generative AI-mediated Socratic instructional package can be an effective and scalable approach for developing epistemic cognition, and they propose a pedagogical model for integrating structured AI-based tutoring into teacher preparation systems aligned with international assessment standards. These findings should be interpreted as evidence for the effectiveness of the structured, AI-mediated Socratic instructional package as a whole, rather than as evidence of AI's inherent superiority over other methods; the relative contribution of the AI medium versus the structured pedagogical protocol it delivers was not directly tested in this design.