Multi-Agent Reinforcement Learning for Energy-Efficient Smart Grid Optimization
Abstract
Smart grid incorporates the use of renewable energy, distributed generation, energy storage, and demand response for enhancing efficiency and sustainability. However, smart grid operation is challenging because of uncertainty of renewable generation, fluctuating electricity prices, changing consumer demand, and the requirement for decentralized control across a number of smart grid entities. Scalability, adaptivity and multiobjective optimization over distributed agents are challenging with traditional approaches such as centralized control, model predictive control, or rule based optimization. Considering these issues, this paper presents a Multi-Agent Reinforcement Learning Framework (MARL-SG) for optimizing the energyefficient smart grid. The framework is based on a centralized training (CT) and decentralized execution (DE) approach, where each grid entity (microgrid, substation, storage unit, consumer cluster) is responsible for learning a local policy, but also sharing a centralized critic. While continuous actions (Power dispatch, Storage charging/discharging, Demand response signals) are optimized across agents using a multi-agent twin-delayed deep deterministic policy gradient (MATD3) algorithm. The framework also features reward shaping based on energy efficiency, grid stability, comfort of the consumer and use of renewable energy. Experimental results show energy cost reduction of 84.73%, renewable curtailment reduction of 79.52%, peak load reduction of 67.38%, voltage stability improvement of 91.46% and consumer comfort preservation of $\mathbf{9 3. 2 7 \%}$ in four smart grid scenarios (microgrid cluster, distribution network, virtual power plant, and islanded operation). The proposed framework shows a considerable improvement compared with other baselines, such as centralized DRL, single-agent RL, and MPC, in terms of scalability, decentralized coordination and energy efficiency for the purpose of sustainable smart grid operation.