Hierarchical reinforcement learning with model predictive elevator control for risk-sensitive multi-robot transport scheduling in hospitals

Abstract

Hospitals increasingly deploy autonomous mobile robots (AMRs) to move specimens and medicines between wards and laboratories. These fleets must meet clinical deadlines that differ across request classes, which couples two decisions: which request each robot serves, and the order of its stops. Intra-hospital transport is formulated here as a multi-objective Markov game and solved with hierarchical multi-agent reinforcement learning for transport (HMART). A shared graph-attention encoder feeds a sequential assignment decoder and a route-improvement decoder, a conditional value-at-risk constraint bounds the tail of class-weighted tardiness, and a model predictive control layer coordinates the capacitated elevators. Two instantiations are evaluated: HMART-H, whose decoders are engineered, and HMART-RL, whose assignment decoder is a trained attention network. In a discrete-event simulator of a four-floor hospital with three elevators, at heavy load with twelve robots over twenty paired seeds, HMART-H lowers mean total weighted tardiness from 2359 ± 214 to 29 ± 6 against the strongest dispatching rule, mean work-in-process from 80.1 ± 6.4 to 14.3 ± 1.9, and the ninety-fifth percentile tardiness tail by more than an order of magnitude, while raising the STAT on-time rate from 0.925 ± 0.011 to 0.969 ± 0.007; all differences are significant under Holm-corrected Wilcoxon signed-rank tests. HMART-RL reaches 40 ± 8 and transfers across fleet sizes and building configurations without retraining. These gains are obtained in simulation and require confirmation in a live building.