Automatic Sleep Stage Classification with Subject-Wise Validation: Comparison of Random Forest, XGBoost, and SVM on the Sleep-EDFx Dataset
TL;DR
A reproducible pipeline for automatic classification of sleep stages using single-channel EEG (Fpz-Cz) from the public Sleep-EDFx dataset (PhysioNet), indicating that the performance ceiling is primarily determined by the feature set and the inherent physiological overlap between stages, rather than by the choice of classifier.
Abstract
This paper presents a reproducible pipeline for automatic classification of sleep stages (Wake W, N1, N2, N3, and REM) using single-channel EEG (Fpz-Cz) from the public Sleep-EDFx dataset (PhysioNet), comprising 100 subjects and 175 recordings from two distinct subsets: Sleep Cassette (SC, 78 healthy subjects, home recordings) and Sleep Telemetry (ST, 22 subjects with mild sleep onset difficulty, placebo nights only). A total of 17 features were extracted per 30-second epoch across two domains: time-domain statistics (mean, variance, standard deviation, skewness, kurtosis, and Hjorth parameters) and frequency-domain features via Welch spectral estimation (absolute and relative power in the Delta, Theta, Alpha, and Beta bands, plus the Delta/Alpha spectral ratio). Three classifiers of fundamentally different algorithmic natures were compared (Random Forest, XGBoost, and Support Vector Machine with RBF kernel) under Leave-One-Group-Out (LOGO) cross-validation, preventing any data leakage between individuals. Experiments were conducted in three scenarios: SC alone, ST alone, and the combined dataset. Random Forest achieved the best overall performance, with Cohen’s Kappa of 0.587 on SC and 0.583 on the combined dataset. The convergence of all three classifiers to a similar Kappa range (0.53 to 0.59) indicates that the performance ceiling is primarily determined by the feature set and the inherent physiological overlap between stages, rather than by the choice of classifier.