AI-based personalized learning and academic performance in computing and engineering higher education: a systematic review and meta-analysis of controlled studies
TL;DR
The available controlled studies suggest a potentially beneficial association between AI-based personalized learning and academic performance in computing and engineering courses, but the magnitude and consistency of this association remain uncertain, and causal conclusions are not warranted.
Abstract
The rapid adoption of Artificial Intelligence (AI) tools for personalized learning in higher education has generated growing interest in their effects on student outcomes, yet comparative evidence remains limited. This systematic review and meta-analysis examined whether AI-based personalized learning is associated with differences in objectively measured academic performance, compared with instruction without the AI component, among higher education students. Following PRISMA 2020, Scopus and Web of Science were searched in April 2026. Eligible studies implemented an AI system that adapted content, feedback, pacing or recommendations to learners, included a concurrent or historical comparator group, and reported an objective academic-performance outcome. Of 84 unique records, four controlled studies (420 students), all conducted in computing and engineering courses, met the criteria: one cluster-randomized study, one study reporting random allocation without procedural detail, one non-randomized study with a concurrent control, and one study with a historical control. Risk of bias was assessed with RoB 2 or ROBINS-I, and certainty of evidence with GRADE. A random-effects model (REML) yielded a pooled Hedges' g of 1.01 (95% CI 0.33–1.68); with the Hartung-Knapp small-sample adjustment the interval widened to −0.10–2.11 and included no effect. Heterogeneity was substantial ( I 2 = 89%; τ 2 = 0.41), and the 95% prediction interval (−0.42–2.43) also included no effect. Leave-one-out estimates ranged from 0.80 to 1.22. Two studies were at serious risk of bias, and the certainty of evidence was very low. The available controlled studies suggest a potentially beneficial association between AI-based personalized learning and academic performance in computing and engineering courses, but the magnitude and consistency of this association remain uncertain, and causal conclusions are not warranted. https://osf.io/amxpf/overview .