Evaluating Explainability-Guided Feature Compression for Network Intrusion Detection: A Comparative Study of SHAP, Mutual Information, and RF Gain

Abstract

Network intrusion detection systems (IDS) typically rely on high-dimensional feature spaces that introduce redundancy and computational overhead. While explainable AI (XAI) techniques such as SHAP have been applied to interpret IDS decisions, their utility for principled feature compression remains underexplored. This paper presents a comprehensive evaluation of SHAP-guided feature compression against two established baselines—Mutual Information (MI) and Random Forest Gain (RF Gain)—across three benchmark datasets: NSL-KDD, CICIDS2017, and UNSW-NB15. Three representative classifiers (Random Forest, XGBoost, and MLP) are evaluated under a unified experimental framework. We systematically assess detection performance (F1-score, AUC-ROC) as a function of selected feature count (K = 5, 10, 15, 20). Results demonstrate that SHAP-guided selection achieves a mean feature compression rate of 77.5% while retaining 98.0% of full-feature F1-score on average. Across 10 random seeds with Wilcoxon signed-rank testing, SHAP significantly outperforms both MI and RF Gain on CICIDS2017 (p = 0.002) and NSL-KDD (p = 0.037 vs. MI), while on UNSW-NB15 all three methods perform near-identically (F1 within 0.4 percentage points). SHAP significantly outperforms RF Gain on all three datasets. SHAP-selected feature subsets exhibit high stability across random subsamples (Jaccard = 1.000, Spearman $\rho \ge 0.95$ ). These findings indicate that SHAP-guided feature compression demonstrates competitive and often superior detection quality relative to traditional selection methods, providing empirical guidance for building compact and interpretable IDS deployments.