Research on trajectory tracking control of robotic arms based on reinforcement learning

Abstract

To address the challenges of model uncertainty, environmental disturbances, and the limited adaptability of conventional control methods in industrial robot trajectory tracking, a hybrid trajectory tracking control strategy integrating Deep Reinforcement Learning (DRL) and Proportional–Derivative (PD) control is proposed. In this framework, the Deep Deterministic Policy Gradient (DDPG) algorithm serves as a reinforcement learning compensator that learns joint-space tracking errors and their variation rates online to generate real-time compensation signals, which dynamically adjust the PD controller output and improve tracking performance. To suppress high-frequency fluctuations produced by the learning agent, a first-order low-pass filter is incorporated between the DDPG compensator and the PD controller, ensuring smoother control inputs and enhanced system stability. The effectiveness of the proposed method is validated using a UR5e industrial robot model developed in the MuJoCo simulation environment. A z-axis step-response test and a straight-line weld seam trajectory tracking task are conducted to evaluate dynamic and tracking performance. Experimental results demonstrate that the DDPG–PD hybrid controller achieves faster response, improved error convergence, and higher tracking accuracy compared with conventional PD control. Moreover, the robot end-effector accurately follows the desired trajectory with low tracking error and smooth motion. These findings indicate that the proposed strategy effectively enhances trajectory tracking performance without requiring an accurate dynamic model, while improving adaptability and robustness in complex dynamic environments.