Co-Optimization of Carbon Storage and Enhanced Oil Recovery Under Stochastic Geological and Market Uncertainty via Deep Reinforcement Learning

Abstract

Abstract Carbon dioxide-enhanced oil recovery (CO2-EOR) coupled with geological carbon storage (GCS) has the potential to simultaneously improve hydrocarbon recovery and permanently sequester anthropogenic CO2. Achieving these dual objectives requires operational decision-making under significant geological and economic uncertainty. Time-lapse geophysical monitoring provides field-observable information on subsurface fluid migration and has the potential to inform adaptive reservoir management. We investigate a deep reinforcement learning (DRL) framework for the co-optimization of CO2 storage and enhanced oil recovery using time-lapse gravity monitoring. Our objective is to determine adaptive CO2-water-alternating-gas (WAG) injection policies that maximize economic performance while accounting for geological uncertainty, stochastic oil-price fluctuations, and monitoring costs. Geological uncertainty is incorporated via stochastic porosity and permeability realizations generated at the start of each training episode, whereas market uncertainty is represented by a mean-reverting stochastic oil price model. We trained a Double Deep Q-Network (DDQN) agent using coupled reservoir simulation and forward gravity modelling, with time-lapse gravity observations serving as the system state. We compare policies learned from five monitoring configurations comprising borehole, surface, and hybrid surface-borehole gravity surveys, as well as a fully observable benchmark consisting of the complete pressure and saturation fields in the reservoir. Comparison of the learned policies demonstrates that (1) the proposed DDQN framework successfully learns robust CO2-WAG control policies under simultaneous geological and market uncertainty; (2) incorporating stochastic oil prices produces more variable reward trajectories and slower convergence but results in economically robust decision-making; (3) dense surface and hybrid gravity monitoring substantially improve learning performance relative to sparse monitoring configurations, with performance approaching that of the fully observable benchmark; and (4) with the exception of the borehole-based strategy, the learned adaptive policies consistently outperform conventional fixed WAG strategies involving regular switching between water and gas phases in cumulative net present value (NPV) while simultaneously increasing oil recovery and CO2 storage. These results demonstrate that integrating time-lapse gravity monitoring with DRL provides a practical framework for uncertainty-aware, observation-driven optimization of CO2-WAG operations by jointly accounting for monitoring geometry and cost, geological uncertainty, and stochastic oil-price conditions, thereby advancing intelligent reservoir management for carbon storage and utilization