Enhanced Intrusion Detection System (IDS) Using Machine Learning

Abstract

This study investigates the development and evaluation of a hybrid Intrusion Detection System (IDS) that integrates rule-based detection with machine learning (ML) techniques to address the limitations of standalone approaches in modern cybersecurity environments. Guided by a Design Science Research framework, the study utilized two benchmark datasets, KDD Cup 1999 and CICIDS2017, representing classical and contemporary network traffic conditions. Three ML models, which are Logistic Regression, Random Forest, and XGBoost, were implemented and compared alongside a traditional rule-based IDS. A hybrid model combining rule-based filtering and XGBoost classification was subsequently developed. Results indicate that while rule-based IDS perform adequately in structured environments, their effectiveness significantly declines in complex and imbalanced datasets. Machine learning models, particularly ensemble methods, achieved superior performance across all metrics, with XGBoost demonstrating the highest overall accuracy. The hybrid IDS achieved consistently high recall rates, indicating strong capability in detecting both known and previously unseen attacks while maintaining interpretability through rule-based components. Statis tical validation confirmed that performance improvements of the hybrid model are significant and robust. The findings support the adoption of hybrid IDS architectures as a balanced and practical solution that enhances detection capability, adaptability, and reliability in evolving cyber threat landscapes.