A Controlled Evaluation of Class Imbalance Handling Techniques in Automated EEG-Based Sleep Stage Classification

Abstract

Automated sleep stage classification from electroencephalogram (EEG) signals is a central task in sleep medicine, yet polysomnographic datasets are inherently imbalanced, with the transitional stage N1 typically comprising only a small fraction of epochs. Although class imbalance handling is widely applied, its independent contribution to sleep staging performance remains unclear because balancing techniques are usually embedded within larger deep learning systems and evaluated jointly with them. This study presents a controlled evaluation of four imbalance handling techniques, namely random oversampling, random undersampling, the Synthetic Minority Over-sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN), against an imbalanced baseline, within an otherwise fixed pipeline comprising single-channel frontal EEG preprocessing, eighteen handcrafted time, frequency and nonlinear features, a Random Forest classifier and five-fold subject-wise cross-validation on the Sleep-EDF Expanded database (20 subjects, 39 recordings, 106,107 epochs, of which N1 comprised 2.64%). Resampling was applied exclusively to training folds. The baseline achieved an accuracy of 0.899 and a Cohen’s kappa of 0.795 while correctly identifying only 13.8% of N1 epochs, suggesting that aggregate measures can conceal minority-stage failures. Although several resampling strategies improved N1 recall, random undersampling produced the highest N1 recall of 0.558 while also producing the lowest precision, suggesting a shift in the decision boundary rather than improved class separability. SMOTE achieved the highest observed macro F1-score of 0.693 with the smallest reduction in majority-stage performance, while random oversampling was largely ineffective and ADASYN showed no advantage over SMOTE. Friedman and Wilcoxon tests did not reach statistical significance under the five-fold design, highlighting the limited statistical power of the evaluation. The findings provide evidence-supported, goal-specific recommendations for selecting imbalance handling strategies and reinforce the need for per-class evaluation in automated sleep staging.