Early Detection of Student Dropout Using Ensemble Learning with Recursive Feature Elimination

Abstract

Student dropout in higher education is a persistent problem with real consequences for students, institutions, and the broader economy. This paper presents a machine learning pipeline for early dropout prediction built on the UCI Student Dropout and Academic Success dataset, which contains 4,424 student records across 36 features. Three base classifiers are evaluated: Random Forest (RF), Support Vector Machine (SVM), and K-Nearest Neighbors (KNN), together with two ensemble methods: a Voting Classifier and a Stacking Classifier. To handle class imbalance, SMOTE is applied within each training fold. Recursive Feature Elimination (RFE) then narrows 36 candidate features down to the 15 most predictive. All models are assessed under Stratified 5-Fold Cross-Validation using accuracy, sensitivity, precision, F1-score, and AUC-ROC. The Random Forest classifier comes out on top (Accuracy = 0.8694, Sensitivity = 0.7516, Precision = 0.8267, F1 = 0.7872, AUC = 0.9137), outperforming both ensemble methods tested. SHAP analysis further confirms and refines the RFE-based feature ranking, supporting the use of this pipeline as an interpretable early warning system for identifying at-risk students.