AI literacy for self-regulated learning among pre-service teachers: development and validation of a scale of ethical use, monitoring, motivation, regulation, and evaluation

Abstract

Introduction Existing AI-literacy measures often emphasize broad knowledge, attitudes, application, or ethical awareness rather than how learners use AI within self-regulated academic learning. This study developed and validated the Multidimensional Aspects of AI Literacy Scale for Pre-Service Teachers (MAILS-PT), which assesses Ethical Use, Metacognitive Monitoring (Monitor), Motivational Regulation (Motivate), Behavioral Regulation (Regulate), and Critical Evaluation (Evaluate). Method An initial 60-item pool was reduced to 40 items through expert content review. The scale was administered to 420 pre-service teachers recruited in Spring 2025 for exploratory factor analysis and to a separately recruited sample of 684 in Fall 2025 for confirmatory and validation analyses. Ordinal-data methods, reliability analysis, assessment of convergent and discriminant validity, and gender measurement-invariance testing were conducted. A 15-item AI-Assisted Learning Judgment Quiz was used to examine preliminary concurrent associations with a framework-aligned judgment measure. Results Exploratory factor analysis retained 30 items, six per dimension; the five-factor solution explained 58.3% of the variance, with primary loadings ranging from 0.639–0.829. Confirmatory factor analysis supported the structure, χ2 (395) = 522.90, p < 0.001, CFI = 0.988, TLI = 0.986, RMSEA = 0.022, 90% CI [0.016, 0.027], SRMR = 0.037. Subscale omega values ranged from 0.851–0.889 in the exploratory sample and 0.848–0.874 in the confirmatory sample. AVE values ranged from 0.522–0.583 and HTMT values from 0.029–0.272. Configural, loading, and threshold invariance across gender were supported under the specific ordered-categorical JASP/lavaan parameterization. Ethical Use showed near-zero relationships with the other dimensions. The five subscales jointly explained 41.3% of judgment-quiz variance, R = 0.643, R2 = 0.413, adjusted R2 = 0.409, F (5, 678) = 95.48, p < 0.001. Conclusion MAILS-PT provides preliminary psychometric support for five distinguishable aspects of learning-related AI literacy among pre-service teachers. Subscales should be interpreted separately, with Ethical Use treated as a distinct yet related domain of responsible academic AI use. Quiz associations provide preliminary framework-aligned concurrent evidence, not independent criterion or predictive validity. Further validation across institutions, cultures, and behavioral or longitudinal criteria is needed.