Constraint-Aware Aggregation for Federated Reinforcement Learning in Microgrid Energy Coordination

Distribution Network Optimization with Aggregation and Reinforcement Learning Under Massive Distributed Resources Integration

The integration of large-scale distributed energy resources (DERs) into distribution networks (DNs) brings challenges to the effective control of DNs. In traditional approaches, mathematical or reinforcement learning (RL)-based solution algorithms are commonly used. However, the exponential increase in the number of DERs reduces the effectiveness of these strategies. Mathematical methods struggle to cope with the dynamic uncertainty caused by the high penetration of renewable energy, while RL algorithms relying on global data training may violate multi-agent privacy protocols. This paper proposes a DNs cooperative optimization method based on resource aggregation and RL. To reduce optimization dimensionality and ensure the privacy of resource data, a dynamic aggregation strategy is employed to aggregate a large number of distributed energy resources into aggregated entities, and the adjustable active–reactive power boundaries of each aggregated entity are derived. To fully exploit the regulation capability of DNs, data centers (DCs), as novel devices, are considered as flexible loads. To improve the convergence speed of model training and decision-making accuracy, evolution strategies (ES) and prioritized experience replay (PER) are integrated into the Soft Actor-Critic (SAC) algorithm, respectively. The proposed method is validated on the IEEE 33-bus and IEEE 123-bus systems. The results demonstrate the effectiveness and superiority of the proposed method in ensuring the secure operation of DNs.

AsyncTrader: Asynchronous Multi-Agent Reinforcement Learning for P2P Energy Trading

Energy-Aware Federated Multi-Agent Reinforcement Learning for Constrained Multi-AUV Cooperative Sensing

Cooperative autonomous underwater vehicles (AUVs) provide an effective platform for marine environmental monitoring and offshore energy exploration. However, federated learning in constrained underwater networks is challenged by the deep coupling between scarce high-value observations and heterogeneous non-IID sensing data. Existing methods usually ignore the coupling between physical energy consumption and model learning, or aggregate sparse heterogeneous updates uniformly, which may cause inefficient training and pseudo-convergence. To address these challenges, this paper proposes an Energy-Aware Federated Multi-Agent Learning framework, named Energy-Aware FedMARL. The proposed framework formulates cooperative sensing as a constrained Markov decision process and adopts a MATD3-based multi-agent strategy to guide AUVs toward spatially complementary high-value observations under energy constraints. In addition, a quality-aware aggregation mechanism is developed to emphasize sparse local updates containing more high-value samples, with extensions to validation-gain and contribution-aware weighting. Simulation results show that Energy-Aware FedMARL reduces global validation loss by about 30% compared with MAPPO and 68% compared with MADDPG, while improving hotspot discovery and effective federated participation.

DER Allocation without Load Prediction via Reinforcement Learning

The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.

Safe Multi-Agent Collaborative Learning for Networked Grid Operation Under Power Network Coupling Constraints

Modern grid operation is increasingly a sequential collaborative control problem under renewable uncertainty, storage dynamics, flexible demand, transmission coupling, and carbon-aware corrective redispatch. This paper focuses on sub-hourly dynamic OPF assistance and safe regional redispatch after forecast updates or emergent network stress. We formulate networked grid operation as a constrained decentralized partially observable Markov decision process and propose a safe multi-agent collaborative learning framework. The method integrates dynamic transfer-stress tracing, a dual-view graph encoder over the physical grid and a real-time stress graph, consensus-based dual coordination for globally coupled constraints, and a differentiable safety projection that maps tentative decisions to executable actions. The framework aims to reduce operating cost, load shedding, renewable curtailment, and carbon-relevant corrective burden, while maintaining feasibility with respect to local linearized surrogate limits and empirically reducing violations in the full simulator. Experiments on chronological grid benchmarks evaluate comparison, ablation, robustness, efficiency, statistical evidence, visualization, and generalization, showing improved trade-offs among cost, reliability, safety, and renewable accommodation.

Federated Aggregation via Artificial Bee Colony and Optimal Transport for Distributed Energy Systems

The rapid development of multi-entity distributed energy systems has underscored the importance of source–grid–load–storage coordinated scheduling in improving renewable energy utilization and reducing operating costs. However, traditional centralized scheduling faces privacy risks, computational bottlenecks, and high latency, making it unsuitable for real-time distributed energy management. To address these challenges, this paper proposes a three-tier “device–edge–cloud” federated edge collaborative scheduling framework FedAOT, integrating an improved Artificial Bee Colony (ABC) algorithm with Optimal Transport (OT) for adaptive aggregation optimization. Specifically, an enhanced ABC is proposed to optimize the aggregation weight vector each round, while OT theory constructs optimal mapping relationships among client models, minimizing parameter distribution discrepancies. Simulation results demonstrate that the proposed method dynamically adapts to variations in data distribution and model quality, significantly improving global model accuracy and convergence speed.