Deep Reinforcement Learning for Flexible Job Shop with Multi-AGV Production Systems via Heterogeneous Graph Neural Networks

Abstract

Flexible job shop scheduling with multiple automated guided vehicles (FJSP-AGV) is a challenging production scheduling problem in intelligent manufacturing, where operation sequencing, machine assignment, AGV allocation, and transportation decisions are tightly coupled. Existing exact and meta-heuristic methods can obtain high-quality solutions, but they usually require considerable computational time for large-scale instances. Meanwhile, conventional dispatching rules can make fast decisions but often fail to capture the complex interactions among operations, machines, and AGVs. To address these challenges, this paper proposes an end-to-end deep reinforcement learning framework based on heterogeneous graph neural networks for solving FJSP-AGV. Specifically, a heterogeneous graph is constructed to represent the scheduling state, where operations, machines, and AGVs are modeled as different types of nodes, and their relationships are described by operation–machine and operation–AGV arcs. Based on this representation, a heterogeneous graph neural network is developed to extract scheduling information from different production resources. In particular, a meta-path aggregation mechanism is introduced to capture the complex interaction patterns among operations, machines, and AGVs. The proximal policy optimization algorithm is then employed to train the scheduling policy in an end-to-end manner. Experimental results on public benchmark instances and real-world cases demonstrate that the proposed method outperforms composite heuristic rules and achieves a favorable balance between solution quality and computational efficiency compared with existing state-of-the-art methods. These results indicate that the proposed HGNN-DRL framework is effective for fast and intelligent scheduling decision-making in FJSP-AGV environments.