Grid-Interactive Hyperscale Data Centers: Deep Reinforcement Learning for Joint Workload-Cooling Scheduling to Enable Demand Response and Renewable Integration

Abstract

Driven by artificial intelligence and cloud computing, hyperscale data centers are becoming one of the fastest-growing electrical loads worldwide and are increasingly recognized as a new class of flexible loads capable of supporting demand response (DR) and the integration of variable renewable energy (VRE). However, their two principal control levers—IT workload scheduling and cooling system operation—have traditionally been managed in a decoupled manner, leaving both energy efficiency and demand-side flexibility under-exploited. This paper proposes a deep reinforcement learning (DRL) framework that jointly co-schedules computing and thermal resources so that a hyperscale data center can operate as a grid-interactive flexible load. We formulate the joint problem as a constrained Markov Decision Process and develop an actor-critic algorithm combining Deep Deterministic Policy Gradient with a safety shield mechanism to guarantee thermal constraint satisfaction during both training and deployment. A high-fidelity digital twin simulation environment enables safe Sim-to-Real training. Extensive experiments demonstrate that the proposed approach reduces total electricity consumption by 18-25% compared to baseline controllers, cuts thermal violations by over 90%, and maintains service level agreement compliance, while broadening the controllable power envelope of the facility to provide a technical basis for participating in DR programs and aligning data-center power profiles with renewable generation. The framework bridges IT-side and facility-side control and supports the evolution of hyperscale data centers from passive electricity consumers toward active, grid-interactive participants in renewable-penetrated power systems.