Physics-Consistent Safe Reinforcement Learning With Risk-Adaptive Stochastic MPC for Cyber-Resilient AC Microgrids

Abstract

Learning-based microgrid energy management has matured rapidly, yet existing safe-reinforcement-learning (RL) and stochastic-MPC hybrids typically treat the AC network as a power-balance constraint and the battery as a scalar state-of-charge integrator. Both abstractions discard the physics that determines whether a control action is feasible on an AC distribution feeder, that drives battery health under stress, and that exposes stealthy false-data-injection (FDI) attacks. This paper introduces a physics-consistent policy-gradient framework for AC microgrid energy management in which the linearized DistFlow branch-flow residual and the battery differential–algebraic residual are embedded as a single differentiable term inside constrained Proximal Policy Optimization. A risk-adaptive CVaR stochastic-MPC inner loop, an input-space control-barrier-function safety filter, a coupled electrochemical–thermal–aging digital twin estimated by a joint unscented Kalman filter, and a cumulative-sum (CUSUM) attack detector form supporting machinery; the same physics residual serves as policy regularizer, parametric Lyapunov contributor, and detector input, so the apparent component count overstates the true mechanism count. Under restrictive Assumptions A1–A7 we state sufficient conditions for recursive feasibility, practical input-to-state stability in the post-convergence regime under stationary exogenous inputs, and convergence of the primal-dual iterates to a local KKT saddle point; these are analytical scaffolding rather than deployment-grade certificates. Validation is simulation-based on a modified IEEE 33-bus testbed driven by NREL irradiance and wind data, a Saudi-calibrated load profile, and the SEC time-of-use tariff, against eight baseline controllers plus a recent physics-informed AC-OPF reference. Across <inline-formula> <tex-math notation="LaTeX">$5\!\times \!500$ </tex-math></inline-formula> Monte-Carlo runs the framework attains annual operating cost of <inline-formula> <tex-math notation="LaTeX">${\$}11$ </tex-math></inline-formula>,340 (95% CI <inline-formula> <tex-math notation="LaTeX">$[{11{,}300, 11{,}380}]$ </tex-math></inline-formula>)—a 38.4% (95% CI <inline-formula> <tex-math notation="LaTeX">$[{37.5,39.3}]$ </tex-math></inline-formula>) reduction versus the rule-based baseline and a 3.3-percentage-point improvement over the strongest learning baseline (RL-MPC, 35.1%); maintains bus voltages within ANSI C84.1 limits in over 99.8% of evaluated steps, including under a coordinated three-bus stealthy FDI attack; and lengthens projected battery remaining useful life by 6.9 years under a stationary-duty-cycle extrapolation. Hardware-in-the-loop validation is left to future work; failure modes, deployment tradeoffs, and assumption boundaries are documented.