Behavioral Drivers of ChatGPT-assisted TOEIC Reading: Evidence from an Extended UTAUT Model

Abstract

This study examined the effectiveness of Artificial Intelligence (AI)-powered Teaching Assistants (AITAs), developed through ChatGPT fine-tuning to support underperforming technical university students in Taiwan, in improving their Test of English for International Communication (TOEIC) reading comprehension. These students are required to meet the English proficiency graduation requirement of a minimum TOEIC score of 550. However, many struggle to attain even 200 out of 495 points in the reading section. Employing a quasi-experimental design, this study involved 226 undergraduate students from non-English major departments over a 12-week period. The experimental group utilized AITAs-driven materials, whereas the control group relied on conventional resources. Learning outcomes were evaluated through pre- and post-tests, and the extended Unified Theory of Acceptance and Use of Technology (UTAUT) was applied to identify the factors influencing students’ acceptance of AITAs-driven materials. An analysis of covariance revealed that students in the experimental group achieved significantly higher TOEIC reading scores than those in the control group. The extended UTAUT analysis highlighted that social influence, AI self-efficacy, performance and effort expectancy, and facilitating conditions significantly influenced behavioral intention. Task-technology fit moderated the relationship between behavioral intention and usage behavior. These findings provide robust empirical evidence for the efficacy of AI-driven personalized learning in enhancing TOEIC reading comprehension for university students. Furthermore, the research offers a comprehensive framework for integrating AI tools into language education practices. It addresses the specific challenges and unique educational needs of learners and leverages AI technologies to help language learners meet standardized testing requirements.