Explainable AI via Causal Mediation Analysis of Neural Network Predictions

Abstract

The increasing deployment of neural networks in critical applications necessitates methods for understanding their decision-making processes – a field often referred to as Explainable AI (XAI). Traditional XAI techniques frequently provide post-hoc explanations that lack a solid causal foundation, potentially masking spurious correlations and leading to misleading interpretations. This paper proposes a novel approach to XAI that leverages causal mediation analysis to uncover the underlying causal pathways driving neural network predictions. We treat the neural network's prediction as a mediator variable, allowing us to rigorously assess the relationships between input features and the target variable through this intermediary. This framework provides a more robust and interpretable explanation by explicitly considering causal mechanisms. We demonstrate the potential of this approach through a theoretical exploration and illustrate its benefits with a simplified example. The core claim of this work is that causal mediation analysis significantly enhances the explainability of neural network predictions by revealing the true causal influences.