Structure-Aware Graph Reinforcement Learning for Dynamic Flexible Job Shop Scheduling with Insertions

Abstract

Dynamic flexible job shop scheduling (DFJSP) with job insertions is a representative challenge in modern manufacturing systems, where dispatching decisions must continuously adapt to evolving shop-floor states. In such event-driven settings, effective reinforcement learning hinges on state representations that can simultaneously capture (i) intra-job technological precedence and (ii) inter-job resource contention on shared machines. This paper proposes Structure-Aware Graph Feature Extraction with PPO (SAGFE-PPO), a learning-based dispatching framework for DFJSP under stochastic job insertions. SAGFE-PPO constructs an incremental operation precedence graph to encode stable process constraints under job insertions, and models real-time machine competition to characterize dynamic resource contention at each decision epoch. The resulting structure-aware embeddings are fused with pairwise operation-machine features and optimized using proximal policy optimization to select feasible assignments. SAGFE-PPO consistently achieves lower makespan than representative DRL schedulers on dynamically generated DFJSP instances with Poisson job insertions, while maintaining comparable inference latency. Gantt-chart analyses further reveal enhanced machine utilization and reduced schedule fragmentation under insertion disturbances.