The naturalness of AI tutoring in English speaking practice: a conversation analysis of turn-taking in human-AI interaction
TL;DR
Conversation Analysis is adopted as a methodological framework to examine recorded speaking practice sessions between English as a Second or Foreign Language (ESL/EFL) learners and an AI tutoring system and indicates that current AI tutoring systems, despite advanced language models, have not yet achieved interactional naturalness as defined from a conversation-analytic perspective.
Abstract
Interactional naturalness of human-AI dialogue is a key factor in fostering meaningful communicative engagement. This study addresses this gap by adopting Conversation Analysis (CA) as a methodological framework to examine recorded speaking practice sessions between English as a Second or Foreign Language (ESL/EFL) learners and an AI tutoring system. Drawing on the principles of emergence, indexicality, and recipient design, the study investigates why and how interactional naturalness is violated in AI-mediated conversation. These findings reveal that while the AI tutor produces grammatically well-formed and topically relevant utterances, its conduct frequently diverges from the moment-by-moment, context-sensitive organization that characterizes natural human interaction. Particularly, the AI system fails to consistently align with emergent turn sequences, indexical references, and recipient-designed responses. The results indicate that current AI tutoring systems, despite advanced language models, have not yet achieved interactional naturalness as defined from a conversation-analytic perspective. Therefore, a shift in evaluation criteria from technical accuracy to interactional quality in the design and assessment of AI-based language tutors is called-for.