PGMPO is proposed, a novel learning framework consisting of a simple but effective multi-policy modeling approach that allows a single network to represent multiple decision-makers, and a preference-driven model optimization method that effectively guides policies to learn diverse and specialized problem-solving strategies without the need for explicit reward functions.
Rising energy costs and the increasing share of renewable generation create incentives to align production schedules with dynamic electricity prices and on-site solar generation. We address the resulting Multi-Objective Flexible Job Shop Scheduling Problem by proposing a deep reinforcement learning framework that jointly minimizes makespan and energy cost. A single preference-conditioned policy approximates the Pareto front at inference time, eliminating the need to train separate models for different objective weightings. The agent acts as a hyper-heuristic, selecting among heuristic actions at each decision point, including strategies that intentionally delay operations to exploit periods of lower electricity prices or higher solar generation. Preferences are integrated throughout the network via Feature-wise Linear Modulation, while a dual-critic architecture and a diversity loss preserve preference-specific policy behaviors. We evaluate the approach against NSGA-II and Joined Heuristics on synthetic instances ranging from 10×5×5 to 15×15×15 jobs, operations per job, and machines using normalized hypervolume and inverted generational distance. While NSGA-II performs best on the smallest instances, the proposed approach becomes increasingly competitive as problem size grows and achieves the best results on the largest evaluated instances. These findings indicate promising scalability within the investigated problem range.
Dustin Moreira Simoes, Marvin Brune, Mehmet Ulrich et al.· Applied Sciences· 0 citations
Machine failures, which represent forms of performance degradation, are common in real-world manufacturing systems, however, they are often overlooked in job shop scheduling solutions that primarily focus on complete machine breakdowns. These subtle disruptions can lead to cascading delays and reduced system efficiency. This study proposes a reinforcement learning (RL) framework designed to address the Job Shop Scheduling Problem (JSSP) in environments affected by such failures. Unlike traditional RL-based scheduling models that concentrate on total breakdowns, this work considers more nuanced disruptions, such as processing slowdowns, which frequently occur in practical settings. The proposed framework enhances a Q-learning algorithm by introducing a refined environment that incorporates an extended state representation and a reward function tailored to account for performance degradation. These enhancements enable the RL agent to learn adaptive scheduling policies that minimize makespan while effectively responding to partial machine failures. The framework is validated using the Taillard benchmark dataset across varying levels of disruption and job-machine configurations. Experimental results show that the proposed environment consistently delivers superior scheduling performance compared to baseline models that do not consider machine failures. The findings highlight the framework’s potential to improve scheduling resilience and efficiency in dynamic production environments.
The flexible job shop scheduling problem (FJSP) is a typical challenge in modern manufacturing system scheduling. Classical optimization methods encounter difficulties extracting complex structural information in disjunctive graphs. An end-to-end approach is proposed in which a graph attention network (GAT) is integrated with deep reinforcement learning (DRL) to address the FJSP. The FJSP is modeled as a multi-Markov decision process (MMDP), where operation selection and machine assignment are treated as separate action spaces. A multiview GAT (MGAT) is introduced to mine the relationships between operations and machines in the disjunctive graph via two interconnected graph attention modules that extract structural information to improve the dynamic decision-making ability of the policy network. The probability distributions for operation selection and machine assignment are predicted by the designed policy networks. The policy networks are trained using the multiproximal policy optimization (multi-PPO) algorithm, and the resulting policies are subsequently employed to solve FJSP instances across various scales. Experiments on benchmark instances demonstrate that the proposed method outperforms heuristic scheduling rules and the heterogeneous graph neural network (HGNN) algorithm.
Fuqing Zhao, Bo Zhou, Ling Wang et al.· IEEE Transactions on Systems...· 0 citations
The dynamic job shop scheduling problem (DJSSP) is critical for optimizing production efficiency in intelligent manufacturing systems under dynamic constraints. Traditional approaches, including heuristic dispatching rules (HDRs) and evolutionary hyper-heuristics, often struggle to generalize across dynamic and unseen scheduling scenarios. The advent of large language models (LLMs) offers new opportunities for scheduling optimization, yet their deployment in real-world manufacturing is constrained by data privacy concerns that limit the use of online LLM services and by the restricted inference capability of local small-scale models. To address these challenges, this paper proposes a novel feedback-driven self-evolution framework employing dual-stage knowledge distillation. The framework utilizes a feedback-driven refinement mechanism to distill expert knowledge from advanced LLMs, which is then used to fine-tune the Qwen2.5-Coder-7B model into ModouGPT, a specialized model for generating HDRs. Experimental results indicate that our approach performs better than traditional methods and five state-of-the-art LLM-based evolutionary frameworks, particularly in dynamic and unseen scheduling scenarios. ModouGPT demonstrates competitive performance with online LLMs in complex dynamic environments, while ensuring data privacy. This study provides manufacturing enterprises, particularly small and medium-sized enterprises, with a deployable scheduling system that balances intelligence, privacy, and computational efficiency in dynamic job shop environments. Note to Practitioners—DJSSP involves constant disruptions such as machine breakdowns and dynamic order arrivals. Deploying artificial intelligence (AI)-based scheduling in real manufacturing environments remains difficult: cloud-based LLM services risk exposing sensitive production data, while large-scale local models demand prohibitive hardware investments, placing both options out of reach for most enterprises. This paper presents a practical solution. The proposed framework distills scheduling knowledge from a powerful online teacher model into a compact 7B local model, ModouGPT, deployable on a single consumer-grade graphics processing unit (GPU). All scheduling decisions are made entirely on-premises, with no data leaving the factory floor. Practitioners do not need domain expertise to design HDRs. The system automatically generates HDRs tailored to the specific production environment, covering job processing characteristics and dynamic disturbances. An offline HDR library can be built in advance and updated periodically without interrupting production, and new HDRs can be synthesized in under one minute during live operations. This makes the system immediately actionable for engineers seeking to upgrade scheduling capabilities without relying on external LLM services or specialized algorithm designers.
Jin Huang, Zhengqi Shi, Qihao Liu et al.· IEEE Transactions on Automat...· 0 citations
Experimental results demonstrate that the proposed framework consistently outperforms classical ALNS baselines and several competitive metaheuristic methods in terms of solution quality and convergence speed.
T. M. Nguyen· Annual Conference on Genetic...· 0 citations
When flexible job shop scheduling faces concurrent disturbances such as machine failures and rush orders, worker-centric constraints emphasized under Industry 5.0 must also be satisfied. Existing deep reinforcement learning methods for the Dynamic Flexible Job Shop Scheduling Problem (DFJSP) seldom treat worker workload balance as an explicit constraint, and most depend on static penalty coefficients that are difficult to tune across different scenarios. In this paper, we suggest ALP-PPO, an adaptive Lagrangian penalty-enhanced proximal policy optimization algorithm, for real-time rescheduling under concurrent machine breakdowns and rush orders. We formulate the scheduling environment as a constrained Markov decision process. Worker skill heterogeneity, fatigue accumulation and workload equity are modeled as coupled constraints alongside classical scheduling objectives. By decoupling operation sequencing, machine allocation and worker assignment into coordinated sub-decisions, a hierarchical action space is constructed. Dual Lagrangian multipliers for workload balance and fatigue are updated adaptively during training, so that manual penalty tuning is no longer required. An event-triggered mechanism selects between right-shift and full rescheduling on the basis of a disruption severity index. We employ weighted-sum scalarization of makespan, energy consumption and workload variance during training, and Pareto solution sets are obtained by systematically varying the weight vectors across independent training runs. On extended Brandimarte benchmarks augmented with worker and dynamic event parameters, ALP-PPO delivers superior scheduling performance across makespan, energy consumption and workload variance when compared with Double DQN, Dueling DQN, standard PPO, NSGA-II and MOEA/D, as measured by Hypervolume (HV) and Inverted Generational Distance (IGD) indicators. Ablation studies indicate that the adaptive Lagrangian mechanism reduces constraint violations by more than 40% relative to fixed-penalty alternatives while keeping the primary objectives competitive. An analysis of computational efficiency shows that ALP-PPO completes online inference in under 20 ms per decision step, making real-time rescheduling practically feasible. Generalization experiments on previously unseen instances further validate the transferability of the learned policy. These findings support human-centric intelligent scheduling in Industry 5.0 manufacturing.