A Novel Recurrent Deep Reinforcement Learning Architecture for Nonlinear and Temporally Dependent Control in the Artificial Pancreas

Abstract

Because of the nonlinear relationship between blood glucose and insulin and the temporal dependencies in human metabolism, the artificial pancreas (AP), a device for controlling blood glucose levels in type 1 diabetes (T1D), is limited in its ability to do so. The majority of traditional control algorithms produce less-than-ideal control because they are unable to account for the complexity of blood glucose and insulin levels. In order to capture the intricacies of blood glucose and insulin levels, we present a novel architecture for recurrent deep reinforcement learning (DRL) in this study. This is accomplished by adding a long short-term memory (LSTM) network to the soft actor critic (SAC) algorithm. This enables the controller to preserve a memory state that contains all pertinent historical states, including trends in continuous glucose monitoring, insulin on board, and meal history. The FDA-approved UVA/Padova T1DM simulator and thirty virtual adults are then used to train and test the controller in silico.Exercises, unexpected meals, and fluctuating insulin sensitivity are just a few of the realistic scenarios in which the controller is tested. In comparison to other controllers like feedforward DRL (74.2 percent ± 4.5 percent), model predictive control (73.5 percent ± 4.0 percent), and proportional integral derivative control (68.1 percent ± 5.2 percent), the suggested recurrent DRL controller can achieve 82.4 percent andplusmn; 3.1 percent time in range (TIR), which is defined as blood glucose levels between 70 and 180 mg/dL. In comparison to the feedforward DRL controller, the suggested controller can also lower hypoglycemia—defined as blood glucose levels less than 70 mg/dL—by more than 40%. Scenarios involving unexpected meals and quick adaptation to high insulin sensitivity can be handled by the suggested recurrent DRL controller.