Predicting in-hospital mortality after cardiac surgery using machine learning: temporal validation in the MIMIC-IV database

Abstract

Background Machine learning (ML) models have been increasingly applied to surgical risk prediction; however, their comparative performance and temporal generalizability remain inadequately evaluated. Methods A retrospective cohort study of 9,956 adult patients undergoing coronary artery bypass grafting and/or valve surgery was conducted using the MIMIC-IV v3.1 database. Logistic regression (LR), random forest (RF), and XGBoost models were developed without class weighting and tuned within nested cross-validation, trained on patients from 2008 to 2016 and temporally validated on patients from 2017 to 2019. Demographics, comorbidities, and 13 laboratory values available within 24 h of admission served as predictors. A class-weighted variant with isotonic recalibration was examined as a sensitivity analysis. Results In-hospital mortality was 2.1% in both the training and validation sets. In temporal validation, LR achieved the highest AUC (0.876), followed by RF (0.856) and XGBoost (0.831); the paired difference vs. XGBoost was +0.044 [95% CI (0.002, 0.084)], marginally excluding zero, whereas the difference vs. RF was not significant. Without class weighting, all three models were well calibrated (calibration intercepts near zero). Age, blood urea nitrogen, creatinine, and partial thromboplastin time were the most influential predictors. Conclusions Under a fair comparison with tuned, unweighted models, logistic regression achieved the highest numerical discrimination, but the advantage was small and at most marginally significant. Logistic regression provides an interpretable and well-calibrated model for early-admission mortality risk stratification, whereas these data do not establish that tree-based models are intrinsically less temporally stable.