Safeguarded Reinforcement Learning for Dynamic Environments

Abstract

Purely machine learning‐based control strategies often suffer from the lack of safety guarantees. In this manuscript, we introduce a locally applied fallback controller to safeguard Deep Reinforcement Learning in dynamic environments. The fallback controller takes over whenever a safety criterion would otherwise be violated. We build upon the previously established connection between a funnel controller and Reinforcement Learning. Typically, for this pure combination, one has to provide a predefined reference trajectory, which is used by the funnel controller for orientation. But since this does not allow for a reaction to changes in the environment, the Reinforcement Learning approach loses one of its core strengths. Thus, we show how a global overall reference trajectory can be replaced by locally defined reference trajectories guiding around potential obstacles. These trajectories can be defined even for moving obstacles and enable safeguarded Reinforcement Learning policies in changing environments. We furthermore show that theoretical results about the funnel controller can be extended to our locally defined scenario and thus provide safety guarantees. The whole framework is explained and shown using the example of a vehicle driving on a racetrack with other road users.