Lightweight Optimal Operational Control of Industrial Systems: A Filter-Based Reinforcement Learning Scheme

Abstract

Data storage-based optimal operational control (OOC) methods typically require the persistent excitation (PE) condition to ensure the existence and convergence of the least squares solution. However, this requirement can be overly restrictive in practical industrial environments, where resources are often constrained. On the one hand, the presence of iterative solutions makes it difficult to verify the PE condition online, potentially leading to unnecessary computational delays; on the other hand, the reliance of the PE condition on historical data storage consumes a significant amount of memory resources. To address these challenges, this paper proposes a lightweight OOC scheme that integrates the filtering process with reinforcement learning. Compared to existing OOC methods, the proposed method offers the following advantages: 1) it enables model-free control for unit devices and operational layers; 2) it enables online verification of the finite excitation (FE) condition, thereby avoiding computational delays; 3) it does not require storing historical data, significantly reducing memory resource consumption during online learning. Furthermore, rigorous theoretical analysis demonstrates the convergence of the proposed method. Finally, a numerical example and mixed separation thickening process are provided to validate the effectiveness of the proposed control method. Note to Practitioners—Optimal operational control (OOC) of industrial systems is essential for maintaining key performance indices, such as clean-coal ash content, grinding particle size, and underflow concentration, at desired levels. In recent years, reinforcement learning (RL)-based OOC methods have attracted considerable attention because they can eliminate the reliance on explicit system dynamics. However, existing RL-based OOC approaches require the persistent excitation (PE) condition and historical data storage to ensure parameter convergence. These requirements lead to computational delays and significant memory consumption, which limit their practical implementation in industrial environments with limited computational resources. To address this issue, this paper proposes a lightweight OOC method that integrates RL with a two-layer filtering structure. The proposed method introduces a finite excitation (FE) condition that can be verified online, thereby eliminating the requirement for data storage and PE. Once the FE condition is satisfied, the optimal operational controller can be computed directly from real-time filtered outputs. Finally, the effectiveness of the proposed method is demonstrated through a numerical example and mixed separation thickening process.