Priority-driven reinforcement learning for safe control: addressing internal dependent state constraints in non-affine discrete-time systems

Abstract

This work presents a model-free reinforcement learning methodology for output tracking control of unknown non-affine discrete-time systems with explicit safety guarantees on internal states. The proposed framework employs a fuzzy rule-emulated actor-critic architecture – built on two Multi-Input Fuzzy Rule-Emulated Networks (MiFRENs) – that integrates a zero-offset barrier function within a novel safe-hierarchical reward function. Unlike conventional approaches that incorporate safety penalties through a weighted combination with the tracking objective, which requires laborious manual tuning of competing coefficients, the proposed hierarchical formulation naturally and unconditionally prioritises the satisfaction of internal-state safety constraints at every sampling instant. Two synchronous, real-time learning laws are derived: a time-varying learning rate for the actor network and a fixed learning rate for the critic network, with convergence of the tracking error and all internal signals rigorously established. Experimental validation on a DC motor position control system confirms the method's superiority: the proposed controller achieves precise trajectory tracking while considering the constraint of armature current in accordance to safety bounds. Comparative analysis against two benchmark reinforcement learning controllers demonstrates that the proposed scheme yields the lowest mean spectral entropy for both the internal state and the control effort, indicating narrower probability distributions and substantially enhanced stability and control smoothness.