Optimizing Outpatient Scheduling Using Bayesian Markov Decision Process and Look-Ahead Simulation: A Case Study of Healthcare Service in Taiwan

Abstract

Purpose Our purpose is to study of outpatient services face unpredictable demand, periodic patient overload, manpower constraints, and inefficient facility and energy utilization. Existing digital healthcare solutions emphasize prediction but provide limited adaptive decision support under uncertainty. Methods This study develops and evaluates, entirely in simulation, a dynamic decision-support framework that integrates a Bayesian Markov decision process (Bayes-MDP) with Monte Carlo look-ahead simulation to balance patient traffic, specialist allocation, facility management, and energy use. The novelty lies in combining Bayesian demand forecasting, Random Forest–based overload-risk estimation, and Bayes-MDP planning within a single look-ahead evaluation framework. Using Weekly outpatient data (2007–2025) from Taiwan’s National Health Insurance (NHI) database and real-time outbreak and disease surveillance (RODS), aggregated by county and disease (influenza/influenza-like illness, acute diarrhea, acute upper respiratory and enteroviral infections; 88 county–disease series). A probabilistic time-series model forecast weekly demand; a Random Forest classifier converted demand history and seasonal features into overload-risk probabilities (held-out AUC = 0.940 under chronological validation). A Bayesian Markov decision process over quantile-based demand states, with Dirichlet–multinomial belief updating and 4-week look-ahead, selected among policies (manpower adjustment, rescheduling, telemedicine, and energy-saving strategies). Policies were evaluated over N = 50 Monte Carlo replications with stochastic demand against a static scheduling baseline simulated on identical demand traces, alongside risk-threshold-rule and myopic one-step benchmarks. Results For the results, the bayes-MDP look-ahead policy reduced the mean simulated outpatient queue by 39% (95% CI 38.8–39.3) and estimated (proxy-based) energy consumption by 5.4% (95% CI 5.3–5.4) relative to the static baseline, with energy-saving actions concentrated in low-risk periods (51% of low-risk weeks). Simpler risk-treshold rule achieved by 12.1 and queue reduction, but slightly increase energy use; myopic one-step policy performed comparably to the look-ahead policy. No single fixed action performed best across all risk levels. Conclusion In conclusion, the simulation results suggest that adaptive, risk-aware decision support can improve service level, specialists workload balance, and facility and energy management compared with static scheduling. Because all findings are simulation-based and energy estimates depend on proxy variables, prospective validation into the real-world scenario would be in real clinical settings is required before implementation.