Learning-Based Cooperative Maneuver Planning for Multi-Spacecraft Proximity Operations

Abstract

We address the problem of multi-spacecraft formation acquisition by comparing a traditional LQR+APF controller with a reinforcement learning based agent. The scenario demonstrates maneuvering three deputy satellites under discretized Hill-Clohessy-Wiltshire dynamics. The operational environment imposes hard keep-out and collision-avoidance constraints. For the guidance task, we employ a multi-agent PPO (MAPPO) controller. The training pipeline leverages centralizedtraining/decentralized-execution and is initially warm-started through behavior cloning. In a paired 200-episode hold-out evaluation, the final learned controller achieves 0.805 success versus 0.850 for the LQR+APF baseline. The paired comparison does not detect a statistically significant difference $(p=0.175)$, and neither controller incurs a hard safety violation. The results validate the learning pipeline while demonstrating a reinforcement learningbased controller can perform similarly to LQR and maintain safety regions.