Comparative Study of Deep Q-Network-Based Reinforcement Learning and Model Predictive Control for Zone Comfort and Energy Optimization in VAV-Controlled HVAC

Safe Deep Reinforcement Learning for Energy-Efficient HVAC Control in Multi-Zone Residential Buildings

HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67\% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6\% energy savings. The PPO policy satisfies formal safety certification with a margin of $2.003^\circ$C. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.

Collaborative optimization of car-following control and energy management for PHEBs based on deep reinforcement learning

To address the synergistic optimization of car-following control and energy management, this study proposes a collaborative optimization framework based on deep reinforcement learning (DRL). At the car-following control level, a predictive cruise control (PCC) model is developed using the twin delayed deep deterministic policy gradient (TD3) algorithm, which incorporates safety and passenger comfort and power demand into the reward function. At the energy management level, a TD3-based energy management strategy (EMS) is formulated, incorporating constraints on battery state of health (SOH) degradation, temperature violations, state of charge (SOC) fluctuations, and comprehensive operating costs. Simulation results demonstrate that, compared to the traditional hierarchical optimization framework, the proposed strategy achieves significant improvements in terms of mean absolute jerk, root-mean-square (RMS) value of acceleration, power demand, battery SOH degradation, battery temperature violation, and comprehensive operating cost, with optimization rates of 32.7%, 49.55%, 5.39%, 28.30%, 91.76%, and 22.82%, respectively. Furthermore, generalization validation indicates that the proposed framework maintains strong robustness and adaptability under unknown driving conditions.

Physics-Informed Deep Learning and Rule-Based adjustment PIDL-RB for online Building Energy Management System: A Comparative Study

A Human-AI Teaming Framework for Deep Reinforcement Learning-Based Voltage Regulation in Distribution Networks

The growing penetration of distributed energy resources (DERs) has increased the operational variability of distribution networks, making voltage regulation increasingly challenging. Conventional deep reinforcement learning (DRL) methods exhibit unsafe exploration behavior, slow convergence, and limited reliability, which restrict their applicability in safety-critical power system settings. This paper presents a human-interactive reinforcement learning framework that enhances the safety and robustness of autonomous voltage regulation. The proposed approach integrates a Soft Actor-Critic (SAC) agent with an adaptive Lagrange constraint mechanism to enforce voltage limits, while a human-guidance module provides sensitivity-based corrections through coordinated capacitor-bank and Battery Energy Storage System (BESS) dispatch. These corrections are incorporated into a human-regularized actor loss, enabling the policy to internalize safe and interpretable control behavior. The framework is implemented in the PowerGym-OpenDSS environment using the IEEE 13-node feeder. Simulation results show that the proposed Human-Interactive Lagrangian SAC (HI-LSAC) achieves significantly lower voltage-violation severity and reduced power losses compared with the other baseline methods.

A deep learning based predictive control method for enhancing microgrid resilience

Integrated Deep Reinforcement Learning Framework for Adaptive PI Control and Multi-Objective Energy Management in Electric Vehicle Powertrains

Electric vehicle (EV) powertrains involve complex interactions between speed regulation, energy consumption, regenerative braking, and battery thermal behavior. Most existing approaches address controller tuning and energy management separately, which may limit the overall system performance. This paper proposes an integrated deep reinforcement learning (DRL) strategy in which a single Twin Delayed Deep Deterministic Policy Gradient (TD3) agent simultaneously adjusts the proportional and integral gains of the speed controller (Kpv, Kiv), the torque modulation coefficient (Ks), and the regenerative braking factor (βreg). A multi-objective reward formulation is adopted to account for speed tracking performance, energy efficiency, regenerative energy recovery, battery thermal constraints, and driving comfort. The framework is implemented through a MATLAB R2022b/Simulink–Python 3.10 co-simulation environment that enables online interaction between the EV model and the learning agent. Performance is evaluated using the Worldwide Harmonized Light Vehicle Test Procedure (WLTP). Compared with a conventional fixed-gain PI controller, the approach reduces gross energy consumption by 16.2%, decreases speed tracking error by 43.7%, increases regenerative energy recovery by 21.4%, limits battery temperature rise by 30.4%, and lowers RMS jerk by 33.7%. The results indicate that jointly optimizing control and energy management variables can improve both vehicle dynamic performance and energy utilization. The methodology offers a practical framework for the development of adaptive and intelligent control systems in future electric vehicles.