Graph-based reinforcement learning framework with control barrier functions for safe marine traffic control

Abstract

Centralized control that optimizes ship behavior is an effective approach for managing marine traffic in congested waters. This study proposed a decision-making framework that generates optimal control actions for all ships in a sea area using traffic data represented as a graph. In this framework, control actions that optimize overall marine traffic are learned through reinforcement learning (RL). Safety constraints based on control barrier functions (CBFs) are introduced to prevent local dangers from being overlooked during optimization. To incorporate CBFs into RL, this study proposed a deterministic graph control barrier function (D-GCBF) that does not require explicit state equations and is independent of the policy. D-GCBF deterministically evaluates whether a policy violates the safety constraints while implicitly accounting for future maneuvering behavior. D-GCBF is integrated into the RL objective function. A centralized control policy trained using this framework was evaluated through numerical experiments in both simple and congested scenarios. The results indicate its ability to control all ships safely while maintaining larger safety margins. Compared to policies trained using RL or D-GCBF alone, the proposed framework outperformed all tested approaches. Specifically, it prevents situations where ships become surrounded by multiple ships and cannot perform collision-avoidance maneuvers. Furthermore, comparisons with a decentralized collision-avoidance artificial intelligence and a representative Lagrangian-based safe-RL method demonstrated that the proposed framework reduces the occurrence of dangerous situations while maintaining efficient navigation toward the waypoint. These findings indicate that the proposed framework could help realize next-generation maritime traffic management systems and enhance marine transportation safety.