SCRAMBLE: Safe Cooperative Risk-Aware Multi-Agent Belief Learning for Embodied Reinforcement Learning in Partially Observable Dynamic Environments
Abstract
Safe decision-making in partially observable dynamic environments remains a fundamental challenge for multi-agent embodied reinforcement learning, where agents must cooperate with limited local observations while responding to evolving environmental changes and safety-critical interactions. Existing methods often emphasize coordination efficiency or reward maximization, but they are less effective at jointly handling latent state uncertainty, dynamic non-stationarity, and safety constraints in a unified learning framework. To address this issue, we propose SCRAMBLE, a safe cooperative reinforcement learning algorithm for multi-embodied agents operating in partially observable dynamic environments. SCRAMBLE combines cooperative belief modeling, risk-aware safety estimation, and constrained policy optimization to enable agents to infer hidden environmental dynamics, anticipate unsafe interactions, and adapt their joint behaviors under changing conditions. Specifically, the cooperative belief module fuses local observation histories and inter-agent information to construct a shared latent representation for decentralized decision-making. On top of this representation, a risk-aware safety estimator evaluates potential hazards caused by environmental variations and multi-agent interactions, allowing the policy to proactively avoid unsafe actions. Meanwhile, the constrained optimization mechanism improves task performance while maintaining safety compliance during learning and execution. Through this integrated design, SCRAMBLE enhances both cooperative robustness and safety adaptability in complex embodied scenarios. Experimental results in representative partially observable dynamic benchmark tasks show that SCRAMBLE consistently achieves superior performance in cumulative reward, safety satisfaction, and adaptation capability compared with existing baseline methods.