Towards Trustworthy Intrusion Detection: A Calibration-Aware and Explainable Classical-ML Pipeline on NSL-KDD

Abstract

Existing assessments of machine-learning-based intrusion detection systems (IDSs) often consider each of accuracy, reliability of probability outputs, and transparency of decisions separately. Few attempt to combine all three dimensions in a single assessment. This paper provides a reproducible IDS pipeline that satisfies all three dimensions simultaneously using four classical algorithms: Logistic Regression, Linear Support Vector Machine, Gaussian Naive Bayes, and Random Forest, appropriate for tabular network-flow data. The pipeline encompasses feature conditioning, model training, probability recalibration, and interpretability. Categorical data are one-hot encoded, and continuous data are Z-score normalized, with the same transformations applied to the NSL-KDD data. For model training, we use the KDDTrain+ set, and we evaluate the final model on the KDDTest+ set using various classification metrics and calibration. Among all proposed models, Random Forest had the best overall metrics, achieving 0.989 accuracy and 0.990 F1 for the attack class, and the best calibration with a Brier score of 0.009. The research shows that classical models, when properly calibrated and made interpretable, can serve as competitive benchmarks for IDS systems. Future research should test this pipeline on other datasets and consider the effect of interpretability on real-world calibrations, such as latency.