Prospective internal validation of an XGBoost-based machine learning model for sepsis prediction in patients with prolonged ICU stays

Abstract

Introduction: to prospectively validate a previously developed machine learning model for predicting sepsis onset in this patient population. Methods: Prospective validation was conducted in accordance with TRIPOD+AI recommendations using data from the Russian Intensive Care Dataset (RICD v3.0). The XGBoost-based model estimated the risk of sepsis onset on an hourly basis with a 6-hour prediction window, using demographic, clinical, laboratory, and vital parameters aggregated into dynamic time windows. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC) with a 95% confidence interval (CI), as well as diagnostic metrics at the optimal cutoff point. Results: Validation was performed on a cohort of patients admitted to the ICU of the Federal Research and Clinical Center of Intensive Care Medicine and Rehabilitology between October 2024 and September 2025 (n = 103; 53,561 patient-hours). The median ICU length of stay was 32 (27; 38) days. The XGBoost model achieved an AUROC of 0.737, consistent with the internal testing results (AUROC 0.752; 95% CI 0.730; 0.767). The optimal cutoff (≥0.5651) yielded a sensitivity of 60.3%, specificity of 81.5%, and a high negative predictive value (97.4%). Although the model showed a high rate of false positives, it demonstrated strong clinical utility for ruling out sepsis within the next 6 hours. Moreover, a multilevel interpretation approach allows risk stratification across low, intermediate, and high probability zones of sepsis development. Conclusion: Prospective validation confirmed the stability and reproducibility of the machine learning model for sepsis prediction in patients with prolonged ICU stays. The results support the feasibility of integrating the model into an electronic health system as a component of a decision support framework for subsequent real-world evaluation.