Large Language Model Driven Reinforcement Learning based Intelligent Control Method for Industrial Cyber-Physical Systems

Abstract

Controllers which can interpret and react to high-dimensional process state and perturbations without loss of stability are needed for industrial cyber-physical systems. In this paper, we propose a large language model driven reinforcement learning approach based on semantic state augmentation using the Llama-3-8B model and a Proximal Policy Optimization controller. Process numerical values are converted to structured operation descriptions and concatenated with normalized feature vectors from sensors to provide the policy with both dynamics of the process and relationships between its components. We create an environment for experiments using Siemens Process Simulate software and Python 3.11 with Tennessee Eastman Process data. As a result, we demonstrate that the proposed controller achieves the normalized control RMSE of 0.158 ± 0.014 versus 0.223 ± 0.018 for classical PPO and a settling time of 8.47 ± 0.71 s versus 11.36 ± 0.94 s. Under combined disturbances, the proposed controller achieves a recovery time of 12.8 ± 1.1 s and a cumulative reward of 846.7 ± 25.8, demonstrating improved control accuracy and disturbance adaptability while maintaining computational feasibility.