Gas-Flow-Rate Control Method of Solid-Rocket Ramjet Engines Based on Deep-Reinforcement Learning

Abstract

The solid-rocket ramjet is a promising and critical propulsion component for propulsion systems in hypersonic vehicles. However, the gas-flow regulation system of the solid-rocket ramjet exhibits strong nonlinearity, time-varying parameters and abrupt operating-condition changes, which makes it challenging to design a qualified gas-flow controller. Despite rapid advances in the field of modern control theory, successful gas flow controllers that could deal with anti-regulation and response time are still lacking. To mitigate these drawbacks, this paper proposes a novel intelligent control method integrating a deep-reinforcement-learning framework with a Tracking Differentiator–PID (TD-PID) controller, aiming at suppressing anti-regulation while achieving fast response. Specifically, a mathematical model with gas-flow regulation is first devised to describe the anti-regulation characteristics of the gas-flow system. Next, the Soft Actor–Critic (SAC) is employed to adaptively tune the parameters involving the speed factor of TD-PID and effectively achieves fast tracking and suppresses anti-regulation. Meanwhile, by adopting staged training and replacing model mapping with action mean values, the proposed approach overcomes the influence of continuous system characteristic drift on training and adapts to scenarios with unknown pressure commands. The proposed method can suppress anti-regulation without sacrificing the response time by adjusting weight coefficients and adaptively optimize TD-PID parameters according to real-time operating conditions. Simulation results demonstrate that the proposed method exhibits superior performance in terms of response speed and overshoot suppression and requires far fewer simulation episodes than its counterparts, which provides a reliable intelligent control solution for gas-flow-rate regulation systems of solid rocket ramjet engines.