Learning When to Observe: A Reinforcement Learning Strategy for Exploration Scheduling in Mobile Robot Systems

Abstract

Accurate station-state information is essential for reliable fleet control in autonomous warehouse systems, but fixed sensing infrastructure is costly and difficult to scale. This paper studies exploration scheduling for warehouse station monitoring under partial observability, where autonomous mobile robots are used as opportunistic sensing agents to maintain the controller's belief about pallet-station occupancy. We formulate the problem as a trade-off between belief accuracy and observation cost and compare three strategies: no explicit re-observation, an aging-based revisitation heuristic and a reinforcement-learning-based scheduler. To support adaptive scheduling, we propose a conditioned policy architecture that combines station-specific long-term context with short-term monitoring features and updates online using Proximal Policy Optimization. Evaluation is performed in a replay-based simulation using recorded robot trajectories and ground-truth station data for 100 stations over three operating periods. Immediate observations are assumed for all methods, isolating exploration scheduling from path planning and execution effects. The results indicate that, among the evaluated methods, the proposed reinforcement learning approach achieves the best trade-off between belief accuracy and observation effort. In the evaluated setting, it reduces the average state-belief error from 2.73 to 1.30 hours per day and per station while requiring fewer explicitly scheduled explorations than the aging-based baseline. These findings demonstrate that adaptive exploration scheduling can improve information freshness and monitoring efficiency in a warehouse.