Evaluating classification rules for predicting local hospitalization surges exclusively from simulated pandemics.

Abstract

Hospitalization dynamics during pandemics caused by novel pathogens are highly heterogeneous across communities, shaped by local contact patterns, the adoption of risk-mitigating behaviors and policies, and baseline population health, including comorbid conditions associated with severe disease. Yet, most existing forecasting approaches rely on national or state-level trends and require the accumulation of surveillance data. This limits the usefulness of these prediction models for local predictions, especially when local dynamics is different from the national or state-level dynamics due to, for example, the emergence of novel strains. We evaluate whether classification models trained exclusively on simulated pandemic trajectories, without exposure to real-world outbreak data, can predict whether hospital occupancy will exceed a prespecified threshold 3 weeks in advance. Using a transmission model, we generated simulated pandemics spanning a wide variation in pathogen characteristics, including transmissibility, virulence, and variant emergence, as well as population features such as age structure and the timing, duration, and effectiveness of control measures. We used these simulated trajectories to train classifiers that predict whether hospital occupancy would exceed a predefined local capacity threshold three weeks in advance. When applied to COVID-19 surveillance data from 799 U.S. Health Service Areas between July 2020 and July 2022, simulation-trained models achieved predictive performance comparable to models retrained weekly using accumulated surveillance data. Simulation-trained random forest models attained area under the ROC curve values exceeding 0.8 throughout the study period and demonstrated robust performance across multiple evaluation metrics, including the area under the precision-recall curve, sensitivity, and specificity. By demonstrating that classification models trained exclusively on simulated data could predict surges in local hospitalizations three weeks in advance, these findings offer a complementary tool to support locally informed decision-making during data-scarce phases of pandemics.