Explainable AI for System Security

Abstract

This paper investigates the application of Explainable Artificial Intelligence (XAI) to enhance system security. Traditional security systems often rely on opaque "black box" AI models, hindering effective threat detection and defense. This work proposes a system architecture leveraging XAI techniques to provide interpretable insights into AI-driven security decisions. The core claim is that building an XAI-based system for security threat detection and defense will improve both the efficiency and reliability of security measures. The proposed mechanism utilizes XAI to elucidate the reasoning behind AI's judgments, thereby facilitating the identification of vulnerabilities and potential threats. Specifically, we explore methods for generating explanations that highlight critical factors influencing security decisions, allowing human analysts to validate, refine, or override AI recommendations. This approach addresses the limitations of current black-box AI security systems by incorporating human understanding and control, ultimately leading to a more robust and trustworthy security posture. The research contributes to a paradigm shift in system security, moving beyond solely relying on AI's predictive capabilities to actively incorporating human expertise within the decision-making process.