A safety-constrained deep reinforcement learning strategy for velocity control of a pipeline inspection robot.

Abstract

Maintaining stable velocity control for a self-propelled pipeline inspection robot remains challenging under high-dimensional state spaces, time-varying friction, and non-stationary operating conditions. This study proposes a safety-constrained deep reinforcement learning control strategy for robust velocity regulation under contact-friction and lubrication disturbances. The proposed method employs a quantile-based distributional critic with a Huber loss to improve value estimation under uncertain disturbances. A lower-tail distributional risk indicator is introduced to regulate curiosity-driven exploration and clipped policy updates. In addition, a safety shield projects raw actions into the feasible actuator range, and the shield correction cost is fed back into policy learning. The controller is trained in Isaac Sim and deployed on an experimental prototype. Experiments are conducted under different operating conditions and velocity disturbances. Compared with the A2C baseline and ablation variants, the proposed method achieves lower MAPE under all tested conditions, with an average reduction of at least 16.5%. Compared with alternative value- and policy-update variants, the proposed method reduces the average MAPE and IQR by 67.4%-76.0% and 41.3%-62.7%, respectively, while exhibiting lower overshoot. These results indicate that the proposed method improves tracking accuracy, reduces velocity dispersion, and enhances velocity-control robustness under the tested contact and lubrication disturbances.