AI-supported assessment in online learning: a Chinese- and English-language systematic scoping review and evidence map of computational approaches, assessment-use impact, and technical validation
TL;DR
This systematic scoping review and evidence map examined computational approaches, assessment-use impact, publication-language patterns, and technical validation evidence in Chinese- and English-language reports published from 1 January 2020 to 31 May 2026, finding that the literature is dominated by Medium-impact diagnostic and decision-support uses.
Abstract
Artificial intelligence supports feedback, diagnosis, prediction, intervention, and scoring in online learning, yet technical performance alone does not establish how an AI-generated output should be interpreted or used as assessment evidence. This systematic scoping review and evidence map examined computational approaches, assessment-use impact, publication-language patterns, and technical validation evidence in Chinese- and English-language reports published from 1 January 2020 to 31 May 2026. The search programme covered 13 structured information sources, supplemented by additional retrieval and citation tracking. Retained search and export records documented at least 7,969 retrieval/export occurrences before source-level relevance filtering and cross-source consolidation. After consolidation, 436 identifiable candidate report instances entered report-level reconciliation; one residual exact duplicate was removed, leaving 435 unique reports for eligibility assessment. Of these, 378 were included and 57 excluded. The individual report (publication) was the unit of analysis. Among the included reports, 99 were Chinese-language (26.2%) and 279 English-language (73.8%). Classical machine learning/data mining, deep learning/neural models, and learning analytics/statistical modeling together accounted for 68.3% of the evidence base. Assessment-use impact was concentrated in decision-support applications: 45 reports (11.9%) were Low impact, 323 (85.4%) Medium impact, and 10 (2.6%) High impact. LLM/generative AI accounted for six reports (1.6%). A focused technical subset of 15 High-impact and/or LLM/generative AI reports underwent task-specific performance synthesis and seven-domain technical validation appraisal. Eleven reports contained sufficient evidence for scoring (three Substantial, six Moderate, and two Limited), while four remained source-limited and unscored. Several reports documented strong task-specific performance or human–machine agreement, but no independent external validation was identified in the available evidence for the focused subset. Overall, the literature is dominated by Medium-impact diagnostic and decision-support uses. Task-level performance, internal validation, and cross-context generalizability represent distinct layers of evidence, supporting a use-sensitive approach to assessment-use calibration.
