State-Space Model-Enabled Reinforcement Learning for Magnetic Configuration Controlon EXL-50U

Abstract

Accurate feedback control of the plasma current ($I_p$) and centroid position $(R_c,Z_c)$ is essential for the stable operation of spherical torus (ST) plasmas. Conventional proportional-integral-derivative (PID) controllers require extensive manual tuning and struggle with the fast, strongly coupled dynamics that arise as plasma performance improves. Reinforcement learning (RL) has recently emerged as a promising alternative to such complex magnetic control problems, yet its practical deployment on ST devices remains challenging. This paper presents a practical RL controller for the EXL-50U ST, trained within a rigid RZIP state-space model (SSM) that enables efficient offline policy learning. A lightweight plasma position reconstructor is developed to estimate $(R_c,Z_c)$ from magnetic probe signals within the real-time control cycle. The trained policy is seamlessly deployed on the EXL-50U plasma control system, achieving stable regulation of $I_p$ and $(R_c,Z_c)$ and sustaining discharges up to 650 ms under RL control. These results demonstrate the feasibility and practical potential of model-informed RL for magnetic configuration control in ST devices, offering a promising direction beyond conventional PID-based schemes.