Centralized Reinforcement Learning for Shared-Target Tracking of Dual Soft Robotic Manipulators with B-Spline-Parameterized Distributed Actuation

Abstract

Coordinated control of multiple soft robotic manipulators is challenging because continuous deformation, distributed actuation, and nonlinear dynamics increase both modeling and policy-learning complexity. This paper proposes a simulation-based reinforcement learning framework for coordinated shared-target tracking using two soft manipulators. Each arm is modeled as a three-dimensional shearable Cosserat rod in PyElastica. A centralized Markov decision process integrates the states of both manipulators and the motion of the shared target. To reduce action dimensionality while retaining distributed actuation, the normal and binormal torque profiles are parameterized by low-dimensional B-spline control points. A cooperative reward penalizes the tracking errors of both arms and encourages simultaneous convergence, while a Twin Delayed Deep Deterministic Policy Gradient algorithm is used for policy training. The policy is trained with bounded random target motion and evaluated on circular, rectangular, and figure-eight trajectories. Simulation results show that both end-effectors track targets with different motion characteristics, achieving mean spatial errors of 0.043–0.065 m. The framework maintains stable tracking under smooth motion and recovers rapidly near sharp turns. These results demonstrate the feasibility of coordinated shared-target tracking for dual soft manipulators and provide a foundation for collision-aware coordinated tracking and simulation-to-real transfer.