Hierarchical AI-Based Deep Reinforcement Learning for Multi-Objective Water and Heat Optimization in Sustainable Data Centers

Abstract

Due to the high rate of artificial intelligence (AI) technology deployment over the last few years, the computational complexity of the current data centers has reached levels that have necessitated higher cooling demands, increased rates of freshwater consumption, and vast quantities of unused thermal energy. The majority of the currently available data center optimization strategies are aimed at reducing the power consumption of the data center. Nevertheless, limited research has focused on cooling water management and waste heat reuse in data centers. The following paper presents a multi-objective optimization algorithm grounded on Deep Q-Network (MO-DQN) to operate and manage data centers of green nature. The operational conditions of the data center are noted as the Markov Decision Process (MDP) in which the states represent the workload demand, thermal condition, cooling capacity, and water availability. The reinforcement learning algorithm, via the interaction with the simulated environment of the data center, will learn the best policies to use in the workload scheduling and cooling systems of the data center to minimize the Water Usage Effectiveness (WUE) and maximize the reuse efficiency of the wasted heat without sacrificing the necessary performance of the computing systems. The presented experimental analysis based on the simulation proves that the proposed MO-DQN framework yields a 26.5% decrease in Water Usage Effectiveness (WUE), 133% increase in waste heat recovery efficiency, and 14% decrease in total energy usage in contrast to the traditional scheduling and single-purpose reinforcement learning strategies.