Risk-Constrained Authority-Coordinated Multi-Agent Reinforcement Learning for Safe and Energy-Efficient Drilling Optimization

Abstract

Abstract Drilling-parameter optimization requires coordinated improvement of drilling efficiency and energy utilization while maintaining engineering safety under strongly coupled and time-varying operating conditions. This study proposes a Risk-Constrained Authority-Coordinated Multi-Agent Reinforcement Learning (RCAC-MARL) framework for the cooperative optimization of weight on bit (WOB), rotary speed (RPM), and flow rate. The draw works, top-drive, and mud-pump systems are modeled as heterogeneous cooperative agents within a centralized-training-decentralized-execution architecture. A causal Transformer reconstructs the partially observable drilling state from multi-source historical measurements, while separate centralized critics explicitly characterize long-term drilling benefit, energy-load cost, and engineering risks. To improve the reliability of offline policy learning, joint-action support regularization constrains extrapolation toward historically unsupported WOB-RPM-flow-rate combinations. A risk-budgeted authority allocator further adjusts the admissible control freedom of individual actuators according to their marginal drilling contribution, energy response, and remaining safety margin. At execution time, a bootstrap probabilistic response ensemble evaluates short-horizon uncertainty and risk, and candidate commands are projected onto a feasible domain satisfying equipment limits, hole-cleaning requirements, probabilistic safety constraints, and historical data support. The framework was evaluated using multi-source field data from 23 drilling wells in Xinjiang, China, with six wells reserved for independent testing. RCAC-MARL achieved an average ROP of 12.97 m/h and an average energy consumption of 28.60 kWh/m, outperforming the strongest Transformer-based baseline, which achieved 12.38 m/h and 29.54 kWh/m, respectively. Moreover, the decision-weighted risks associated with stuck pipe, overflow, and vibration were reduced by an average of 30.28%. The results demonstrate that RCAC-MARL improves drilling efficiency and energy utilization while systematically shifting control decisions away from high-risk operating regions, providing an effective supervisory-control framework for safe and energy-efficient intelligent drilling.