Multi-Armed Bandits for Safety-Aware Network Scheduling Problems under Uncertainty

Abstract

Network scheduling must balance performance with safety considerations, particularly in critical infrastructure systems. While traditional approaches either rely on detailed system models or simplified binary safety indicators, both have limitations in uncertain environments. We propose a novel model-free approach for decentralized network scheduling that incorporates continuous safety indicator measures for each schedule, enabling more nuanced safety assessment. We specifically utilize multi-armed bandits with each possible schedule representing an arm. Our method demonstrates superior performance in identifying and eliminating unsafe schedules while maintaining probabilistic guarantees in finite time. Theoretical analysis reveals important trade-offs between time taken to eliminate unsafe schedules, number of rounds needed and the mean safety levels. We validate our approach through numerical experiments and a real-world case study of a Water Distribution Network in Tampa, FL, where we optimize restoration scheduling under contamination risks. This work provides infrastructure managers with a practical framework for evaluating resilient strategies under simulated disruptions.