Multi-Agent Reinforcement Learning for Dynamic Parallel-Machine Scheduling with Human–Robot Collaboration and Worker Fatigue

Abstract

This study investigates a dynamic parallel-machine scheduling problem with human–robot collaboration (DPMS-HRC), where dynamically arriving jobs are assigned to eligible workstations and processed by workers, robots, or human–robot collaboration based on real-time system states. Worker-involving modes improve operational flexibility but may increase actual processing times and cause uneven fatigue distribution due to fatigue accumulation. To coordinate production responsiveness and fatigue management, a weighted objective combining average flow time and the worker fatigue balance index is minimised. A mixed-integer nonlinear programming (MINLP) formulation is developed to model dynamic job arrivals, alternative processing modes, worker–robot allocation, and fatigue-dependent processing times. An action-masked and potential-shaped multi-agent proximal policy optimisation (AMPS-MAPPO) algorithm is then designed. Action masking restricts policy sampling to structurally feasible job-mode-resource combinations, while time-scaled potential shaping provides dense feedback for long-horizon learning. Experiments on dynamic instances of different scales show that AMPS-MAPPO achieves millisecond-level online inference, reduces average flow time by 40.98% relative to the best dispatching-rule benchmark and by 8.17–20.92% relative to deep reinforcement learning baselines, and obtains the lowest overall worker fatigue balance index among the compared deep reinforcement learning algorithms.