Insights into Faculty Members’ AI Literacy and Emerging Pedagogical Mindset Change Towards Assessment as Learning

Abstract

AI is influencing how language is learned and taught and, thus, how language learning is assessed. This study explores the triadic relationship among AI literacy, mindset transformation (MiT), and assessment as learning (AaL) to redress AI-revamped language assessment forms in AI-driven language education. It elucidates how faculty members’ AI literacy is associated with their pedagogical MiT, which, in turn, supports AaL. Towards this end, data were collected in two rounds within an explanatory mixed-methods approach. In the first round, AI literacy and MiT of sixty-four faculty members were surveyed and correlated. In the second round, a subset of the participants was singled out for a focus group discussion. The results showed that faculty members’ AI literacy is associated with their MiT, and this association is linked to using AI tools for AaL in their teaching. A common pattern in the responses was that faculty members frame AI as a supportive rather than authoritative agent. They expressed concerns about using AI in summative assessment due to uncertainty about how AI works; hence, they tend to avoid using it when grading students. Based on the findings, a model of directional relationship is proposed in which AI literacy appears to be associated with MiT, which, in turn, may facilitate pedagogical shifts toward AaL. The model can be used as a basis for further research to validate the correlation between AI literacy and MiT in the context of language education and assessment, and to demystify how faculty members navigate AI integration in language assessment in various contexts.