Feature Selection Versus Feature Extraction for IoT Intrusion Detection Under Temporal Evaluation

Abstract

Comparative studies of feature selection (FS) and feature extraction (FE) for IoT intrusion detection rely almost universally on random train/test splits, which let temporally adjacent, highly similar records appear on both sides of the partition. This study asks whether such conclusions survive when this evaluation leakage is removed. We reproduce the detection-performance component of the framework of Li et al. on the TON-IoT network dataset — identical preprocessing, correlation-based selection versus PCA-based extraction, five classifiers, and matched dimensionalities K ∈ {9, 22, 33, 47, 77} — varying only the partitioning rule across random stratified, global temporal, and per-class temporal protocols. At K = 9, the reference Decision Tree falls from 86.81% binary macro-F1 under random splitting to 58.04% under the global temporal protocol, in which unseen attack classes constitute 45.6% of the test records. In the multiclass task at K = 47, the same configuration declines from 93.14% to 14.26%, while the mean Matthews correlation coefficient (MCC) across 100 configurations falls from 0.638 to 0.196, a relative loss of approximately 69%. The family-level recommendation is directionally robust: for the single redundancy-oriented correlation selector evaluated here, PCA-based extraction outperforms selection in all three protocols (Wilcoxon over 50 matched pairs per protocol, p < 10⁻⁶). Configuration-level orderings and the identity of the best classifier, however, are protocol-dependent. Established for the specific selector and extractor evaluated, these findings indicate that feature-reduction guidance for IoT intrusion detection should be validated under leakage-resistant protocols, with macro-averaged metrics and MCC reported alongside accuracy.