AMI-Informed Hierarchical Deep Reinforcement Learning–Model Predictive Control for Coordinated EV, PV, and Battery Energy Management in Campus Microgrids
TL;DR
An AMI-to-control pipeline that transforms historical 15 min smart-meter measurements into operational flexibility features and embeds them into a hierarchical SAC–MPC architecture remains computationally feasible across EV penetration, seasonal, and forecast-uncertainty scenarios.
Abstract
This paper proposes an advanced metering infrastructure (AMI)-informed hierarchical energy management framework for coordinated operation of electric vehicles (EVs), photovoltaic (PV) systems, and battery energy storage systems (BESS) in campus microgrids. The proposed two-layer architecture integrates a soft actor–critic (SAC) deep reinforcement learning (DRL) agent in the upper layer with a receding horizon model predictive control (MPC) optimizer in the lower layer. The key novelty is an AMI-to-control pipeline that transforms historical 15 min smart-meter measurements into operational flexibility features and embeds them into a hierarchical SAC–MPC architecture, where the DRL layer provides adaptive coordination and the MPC layer enforces grid, storage, and EV-service constraints. The proposed framework using the real-world Pecan Street data (15 min resolution) of 73 homes across Austin, Texas and California (2014–2019) achieves a 53.1% cost reduction and a 25.7% peak demand reduction when compared with uncontrolled charging, and the proposed framework outperforms MPC-only (50.9%), DRL-only (−5.2%), and rule-based (5.1%) baselines. The statistically significant contributions of network-aware constraints, demand-response activation, and predictive look-ahead horizon are statistically significant (n = 10 independent runs) contributions (p = 0.001). The state representation informed by AMI offers directional cost improvement (+8.4%, p = 0.055) with 11% faster convergence of training. The zero network constraint violation is observed in all evaluation scenarios and the average MPC solve time is around 150 ms, which is much less than the 15 min sampling period. Sensitivity analyses show that the hierarchical DRL–MPC architecture remains computationally feasible across EV penetration, seasonal, and forecast-uncertainty scenarios. However, BESS provided no net economic benefit under the evaluated energy-only TOU tariff, increasing weekly cost by $15.25 and peak grid demand by 14.2 kW. Break-even analysis indicates that demand charges of approximately $9.9/kW per month are required for BESS to become cost-effective in the proxy system, highlighting that storage value depends strongly on tariff design and peak-demand objective formulation.