From Prediction to Personalized Intervention: An Explainable, Lightweight SHAP-Driven Ensemble Framework for Early Academic Failure Risk Assessment
TL;DR
A model to interpret the model globally and instance-level using SHapley Additive exPlanations (SHAP) along with three machine learning algorithms (Random Forest, XGBoost, and Gradient Boosting) in a lightweight, easy to explain academic performance prediction framework is proposed.
Abstract
Early identification of students at risk remains one of the most critical problems in e-learning learning, and most of the prediction models sacrifice accuracy to be interpretable, and are expensive to implement using deep learning architectures that are not feasible in the educational systems with limited resources. This paper proposes a model to interpret the model globally and instance-level using SHapley Additive exPlanations (SHAP) along with three machine learning algorithms (Random Forest (RF), XGBoost, and Gradient Boosting (GB)) in a lightweight, easy to explain academic performance prediction framework. We use three publicly available datasets to test the framework: the UCI Student Performance Dataset (649 records, 33 features), Open University Learning Analytics Dataset — OULAD — (32,593 records, 23 features), and the Kaggle Student Habits Dataset (1,000 records, 20 features). Data leakage is prevented by the Synthetic Minority Oversampling Technique (SMOTE) used during training folds and hyperparameters are tuned using RandomizedSearchCV with five-fold stratified cross-validation. The proposed ensemble achieves 94.7% accuracy, 0.943 F1 score and 0.971 AUC-ROC in 1.8 seconds of CPU training time (470× faster than the state of the art deep learning models LSTM and CNN+LSTM, and without requiring any GPU hardware). At the population level, SHAP is a measure of the most important predictive features of the model and automatically provides actionable intervention recommendations for each at-risk student. The Wilcoxon signed-rank test is used to establish the statistical significance (p<0.01) and an ablation study is used to measure the contribution of each component of the system. It should be deployable on common laptop computers and be based on open and free tools, accessible for institutions with restricted computational resources, and explored in relation to the fairness, privacy and trustworthy-AI requirements for educational deployment.