A Stacking Ensemble Framework with Mutual Information Feature Selection and SHAP for Explainable Network Intrusion Detection

Abstract

The rapid evolution of cyber threats has significantly increased the demand for intelligent intrusion detection systems (IDSs) capable of accurately distinguishing malicious network activities from legitimate traffic. Traditional signature-based IDSs are often ineffective against previously unseen attacks, while many machine learning-based approaches suffer from limited interpretability despite achieving high detection performance. To address these challenges, this paper proposes an explainable network intrusion detection framework that integrates Mutual Information (MI)-based feature selection, a stacking ensemble classifier, and SHapley Additive exPlanations (SHAP). The proposed framework is evaluated using the UNSW-NB15 benchmark dataset. During preprocessing, categorical attributes are transformed using label encoding, highly skewed numerical features are normalized through logarithmic transformation, and robust scaling is applied to mitigate the influence of outliers. Mutual Information is employed to rank feature relevance, and the twenty most informative features are selected to reduce computational complexity while preserving discriminative information. A stacking ensemble model comprising XGBoost, LightGBM, and CatBoost as base learners, with Logistic Regression serving as the meta-learner, is then trained for binary intrusion detection. Experimental evaluation demonstrates that the proposed model achieves an accuracy of 93.40%, precision of 94.50%, recall of 95.22%, F1-score of 94.86%, and an ROC-AUC score of 98.74%. Furthermore, SHAP-based explainability provides both global and local interpretations of model decisions, identifying sttl, sbytes, and synack as the most influential features contributing to attack detection. The experimental results indicate that the proposed framework offers a robust balance between predictive performance and interpretability, making it suitable for practical cybersecurity applications.