Energy-Aware Federated Multi-Agent Reinforcement Learning for Constrained Multi-AUV Cooperative Sensing

Abstract

Cooperative autonomous underwater vehicles (AUVs) provide an effective platform for marine environmental monitoring and offshore energy exploration. However, federated learning in constrained underwater networks is challenged by the deep coupling between scarce high-value observations and heterogeneous non-IID sensing data. Existing methods usually ignore the coupling between physical energy consumption and model learning, or aggregate sparse heterogeneous updates uniformly, which may cause inefficient training and pseudo-convergence. To address these challenges, this paper proposes an Energy-Aware Federated Multi-Agent Learning framework, named Energy-Aware FedMARL. The proposed framework formulates cooperative sensing as a constrained Markov decision process and adopts a MATD3-based multi-agent strategy to guide AUVs toward spatially complementary high-value observations under energy constraints. In addition, a quality-aware aggregation mechanism is developed to emphasize sparse local updates containing more high-value samples, with extensions to validation-gain and contribution-aware weighting. Simulation results show that Energy-Aware FedMARL reduces global validation loss by about 30% compared with MAPPO and 68% compared with MADDPG, while improving hotspot discovery and effective federated participation.