Privacy-Preserving Network Intrusion Detection Using Federated Learning with Explainable AI and an Autonomous SOC Agent Pipeline

Abstract

Centralized intrusion detection systems (IDS) re-quire pooling raw network traffic at a single location, which is a significant privacy liability in domains such as banking, healthcare, and telecommunications, and typically rely on static detection thresholds that produce high false-positive rates as traffic patterns drift over time. This paper presents a two-phase system that closes both gaps. Phase-I is a Federated Learning (FL) based IDS in which five simulated clients, each holding a non-identically-distributed partition of the CICIDS2017 dataset, train a per-client LSTM Autoencoder locally and share only model weights — never raw traffic — with a custom FedAvg aggregator implemented over Python threading. Differential Privacy (DP) noise is added to gradients before aggregation to bound information leakage from any single client's update, and an adaptive thresholding mechanism recalibrates the anomaly-detection boundary using a rolling window of reconstruction errors to address the false-positive gap. Per-prediction explanations are produced with SHAP and LIME. Phase-II extends detection into an autonomous, auditable Security Operations Centre (SOC) agent that consumes alerts from the Phase-I detector and executes a six-stage pipeline — Detect, Retrieve, Score, Draft, Verify, Decide — retrieving supporting context from a MITRE ATT&CK knowledge base through TF-IDF based retrieval-augmented generation, computing a rule-based severity score, drafting an incident report, passing every report sentence through a hallucination guardrail before any action is taken, and finally auto-closing, queuing, or escalating the incident. The system achieves a binary classification F1-score of 99.42% (precision 98.94%, recall 99.92%) and a multi-class accuracy of 91.57%, with only a 3.73% accuracy gap between the differentially private and non-private centralized baselines. We describe the system's architecture, workflow, evaluation methodology, and a live walkthrough, and discuss the engineering challenges — including severe class imbalance and framework incompatibility — encountered while building it.