Evaluation of Reinforcement Learning for Autonomous Operation: A Distillation Column Case Study

Abstract

Industrial process autonomy is limited by nonlinear dynamics and the need for operators to handle abnormal events manually. In current practice, economic optimization is managed through a hierarchy of real-time optimization (RTO) and model predictive control (MPC). However, these systems are typically designed for normal operation and rely on human intervention for safety-critical abnormal events. While recent reinforcement learning (RL) studies suggest an alternative to this hierarchy, most use low-fidelity simulators and address control in isolation. This study addresses these gaps by using a high-fidelity CORYS INDISS Plus simulator to implement a dual-agent RL architecture that unifies economic optimization and abnormal event handling within one framework. The first agent manages normal operation during feed-flow disturbances to maximize butadiene and benzene recovery while meeting product quality specifications. This agent acts as a cascade master, adjusting reflux flow, steam flow, and vent flow setpoints. We used the SmoothRecurrentTD3 algorithm, which employs gated recurrent units (GRU) to handle process time delays and smoothing regularizes to reduce valve oscillation. When benchmarked against RTO-MPC baseline on identical cascade setpoints, the RL agent increased combined profit by 9.8% (5256$/h vs 4788$/h). The second agent handles abnormal operations during complete feed loss or lower feed. It uses a safety-oriented reward function to keep reflux and reboiler levels within safe operating bands to prevent pump cavitation and column flooding. This agent was validated over a five-hour test period, with control switched between the economic and safety agents via a feed-flow threshold. RL inference relies on simple function evaluation, which is significantly faster than the iterative optimization required by RTO-MPC. This allows for quicker responses to dynamic disturbances. Field deployment will require further work on agent trust regions, sensor resilience, and adaptation to process drift. This case study provides an empirical benchmark of RL against industrial supervisory control and offers a practical route to reduce operator dependence.