A Reinforcement-Learning-Based Hybrid SAC–LQR Framework for UR3e Multi-Waypoint Motion: System Design and Simulation-Based Evaluation

Abstract

Artificial intelligence is increasingly being investigated for robot motion generation, while conventional methods remain effective for deterministic waypoint tasks. This study evaluates reinforcement learning as a state-conditioned joint-reference-generation layer at runtime, not as a replacement for classical control. A hybrid soft actor–critic (SAC)–linear-quadratic regulator (LQR) framework was implemented for a four-phase UR3e waypoint task. The SAC policy generated bounded joint-reference increments every 50 ms, while six per-joint LQR loops tracked them every 5 ms. Across 50 randomised MATLAB/Simulink episodes, the framework achieved 96% full-task success upon first entry into a strict 5 mm place region, with a first-entry distance of 4.221±0.381 mm. Frozen-policy tests under broader initial configurations, unseen waypoints, model-parameter perturbations, SAC observation-vector noise, command delay, and external torque achieved 82–100% full-task success. Under the nominal protocol, matched offline-trajectory LQR and proportional–integral–derivative (PID) baselines and a conventional inverse-kinematics (IK)–trajectory–LQR baseline each achieved 100% task completion. Post-entry continuation showed that threshold-entry success did not yield sustained sub-5 mm placement over 0.5, 1.0, and 2.0 s dwell intervals. A separate MATLAB–Robot Operating System 2 (ROS 2) fake-hardware experiment characterised execution and communication latency, with a mean round-trip latency of 5.45 ms. It excluded the Simulink plant and LQR inner loops and, therefore, represents a pre-deployment controller-pipeline evaluation rather than validation of the complete architecture on a physical robot.