Adversarial Robustness Evaluation of a Multi-Layer OT Intrusion Detection System Via Automated C2 Red Teaming

Abstract

Deploying machine-learning-based intrusion detection in operational technology (OT) networks requires evidence that the system detects real adversary behavior, not just anomalous packets. Most published evaluations rely on static datasets with atomic attack samples and therefore leave multi-stage robustness, the property that matters when real adversaries chain reconnaissance, credential theft, and process manipulation, unmeasured. This paper evaluates the multi-stage robustness of a multilayer OT intrusion detection system that combines machine-learning anomaly detection, MITRE technique enrichment, Dempster-Shafer evidence fusion, and kill-chain correlation. The evaluation method, Automated Command-and-Control (C2) Red Teaming, drives an adversary emulation platform through repeatable attack chains and treats its execution log as ground truth, removing the manual labeling step that limits prior multi-stage evaluations. Five information-technology-to-OT chains, executed via CALDERA on a physical Modbus testbed, cover 15 MITRE ATT&CK techniques spanning credential theft, lateral movement, and process manipulation. Across 25 runs and 220 ground-truth actions, the system reaches 92.7% true positive rate with 11 of 15 techniques at 100%. Per-technique analysis exposes robustness boundaries that aggregate metrics conceal and identifies the three layers, telemetry, rule, and engine, where targeted hardening must occur.