Scalable RL-Based Online Scheduling Framework for Modular Process Systems

Abstract

To meet the online scheduling requirements of large-scale parallel systems, traditional centralized path planning algorithms suffer from severe computational bottlenecks as system scale increases. This paper proposes FreeFlow, a fully decentralized, expert-free framework for online distributed scheduling. The core highlight of FreeFlow is to decouple decision complexity from system scale, enabling highly parallel execution on ultra-large platforms without the need for global state synchronization. Driven strictly by local observations, FreeFlow employs a guided reward mechanism that promotes proactive collision avoidance and cooperative spatial distribution without relying on expert knowledge or heuristic guidance. To support continuous task streams, we further introduce a dynamic agent pool mechanism based on object reuse, addressing the mismatch between dynamic task arrivals and fixed input dimensions in parallel neural networks. Extensive experiments show that FreeFlow achieves throughput comparable to state-of-the-art centralized methods in large-scale settings while significantly reducing runtime. It also demonstrates strong generalization to larger environments, maintaining over 92% success rate with millisecond-level inference latency in a 1024×1024 grid environment.