Multiliteracies in Action: Using Generative Artificial Intelligence to Develop Digital Multimodal Composing Tasks for Multilingual Learners of English Through Technology‐Mediated Task‐Based Language Teaching

Abstract

Digital multimodal composing (DMC), which includes digital storytelling and writing, has grown markedly with the increasing use of technology in the second language (L2) classroom. Although much of this research is grounded in sociocultural approaches, recent studies have taken a cognitive interactionist approach to explore how DMC can promote noticing and peer interaction. However, few studies exploring DMC have utilized technology‐mediated task‐based language teaching (TMTBLT) as a methodological and pedagogical framework. Addressing calls by previous scholars to explore multiliteracies with DMC task design, this study qualitatively explores how multimodal generative artificial intelligence (GenAI) can support DMC and learners' task engagement. Drawing from a multiliteracies framework, this study examines how 10 upper‐beginning ESL learners used GenAI to generate visuals for meaningful, authentic DMC tasks in which they collaboratively designed artifacts for future students navigating life in Hawaiʻi. Task‐based interactions were recorded and coded for language‐related and prompt‐altering episodes. Additional data sources included chat logs and teacher reflections, which were used for triangulation with the task analysis. Findings suggest that GenAI chatbots can support multilingual learners in fostering critical engagement and enhancing agency in DMC. TMTBLT as a methodological and pedagogical approach promoted learner engagement through GenAI‐assisted DMC.