Refined Environmental Design for Reinforcement Learning Framework in Job Shop Scheduling

Abstract

Machine failures, which represent forms of performance degradation, are common in real-world manufacturing systems, however, they are often overlooked in job shop scheduling solutions that primarily focus on complete machine breakdowns. These subtle disruptions can lead to cascading delays and reduced system efficiency. This study proposes a reinforcement learning (RL) framework designed to address the Job Shop Scheduling Problem (JSSP) in environments affected by such failures. Unlike traditional RL-based scheduling models that concentrate on total breakdowns, this work considers more nuanced disruptions, such as processing slowdowns, which frequently occur in practical settings. The proposed framework enhances a Q-learning algorithm by introducing a refined environment that incorporates an extended state representation and a reward function tailored to account for performance degradation. These enhancements enable the RL agent to learn adaptive scheduling policies that minimize makespan while effectively responding to partial machine failures. The framework is validated using the Taillard benchmark dataset across varying levels of disruption and job-machine configurations. Experimental results show that the proposed environment consistently delivers superior scheduling performance compared to baseline models that do not consider machine failures. The findings highlight the framework’s potential to improve scheduling resilience and efficiency in dynamic production environments.