Interpretable Machine Learning Models for Critical Decision Systems

Abstract

Machine learning (ML) systems are finding more and more applications in high-reliability decision-making setting including medical diagnosis, risk estimation in the finance sector, driverless transport, law enforcement, and fault management in industries. Their lack of transparency makes complex black-box models, especially deep neural networks and ensemble learning methods, highly problematic in safety-critical and high-stakes application areas despite having shown impressive predictive accuracy. The regulatory requirement, ethical concerns, accountability and trust among users enforce that the decisions made by ML systems must be interpretable, clarifiable and verifiable. That caused the increased attention to the area of interpretable machine learning (IML), which is supposed to reconcile predictive score and interpretable reasoning available to humans. This paper is the systematic and complete study of interpretable machine learning models of critical decision systems. We start by examining the conceptual basis behind interpretability and its significance in high risk applications. An elaborate literature review classifies the currently existing interpretability methods as intrinsic interpretability methods and post-hoc explanation methods and their strong and weak points and the appropriateness to critical systems. The suggested methodology describes a systematic approach to the selection, design and validation of interpretable ML models within real-life conditions of uncertainties of data, bias and regulatory standards. Mathematically stated representative interpretable models such as linear models, decision trees, rule-based systems and attention mechanisms are given to provide formal grounding. Examples of experimental findings of representative domains of application are presented to illustrate the trade-offs in interpretability and performance. The discussion highlights levels of interpretability, robustness, and fairness measures, and predictive accuracy. Lastly, the paper draws a conclusion and gives important opinions and future research directions with the aim of achieving credible and open machine learning systems in life and death situations.