Development and Evaluation of an Explainable AI-Driven Decision Support System for Clinical and Operational Healthcare Management
Abstract
Healthcare organizations need to have an integrated decision-support system that can enhance patient risk assessment and optimize their use of resources. But the lack of transparency and separation between the use of AI in clinical use and administrative applications puts adoption challenges in the way.Conceive and evaluate a healthcare management explainable AI-based decision support system integrating clinical-risk prediction, resource-demand prediction and operational healthcare management. A synthetic multicentre dataset of 1,800 anonymized patient encounters was divided into a development set (n=1,260), a validation set (n=270) and an independent test set (n=270). The following models were tested: logistic regression, random forest, support vector machine, extreme gradient boosting, artificial neural network and the proposed explainable AI system. Evaluation of performance was made using the classification, discrimination, calibration, decision-utility, operational, subgroup and professional-acceptance measures. SHapley Additive exPlanations (SHAP) was used for the interpretation of predictions. The proposed system achieved an accuracy of 0.900, sensitivity of 0.730, specificity of 0.952, F1-score of 0.773, ROC–AUC of 0.951, and Brier score of 0.085. Clinical-risk probability, oxygen saturation, comorbidity burden, C-reactive protein, respiratory rate, age and bed occupancy were important predictors. The efficiency of the work process increased by 16.0%, waiting time was reduced by 21.2%, length of stay by 10.9%, and the accuracy of the resource allocation's handling by 14.4% after using it in management. The explainability, trust, usefulness and acceptance were positively reported by healthcare professionals. Explainable AI can enhance collaborative clinical and operational decision making. However, prospective multicenter validation in the real world is needed before clinical implementation.