Multi-Objective Optimization for Data Center HVAC Systems Based on Edge–Cloud Collaborative Deep Reinforcement Learning
TL;DR
This paper proposes an edge-cloud collaborative physics-informed reinforcement learning framework for production data center HVAC control that integrates a physics-informed cold-start solution using Adaptive Particle Swarm Optimization, a three-time-scale edge–cloud architecture, and a constraint-aware safe projection layer that embeds thermal safety hard constraints directly into the neural network policy.
Abstract
The sustained growth of cloud computing and AI training workloads drives data center expansion. Optimizing their control is therefore critical for reducing operational costs. Edge real-time control is indispensable for guaranteeing thermal safety, data sovereignty, and offline availability. Yet deploying Deep Reinforcement Learning (DRL) in production Heating, Ventilation, and Air Conditioning (HVAC) environments confronts cold-start risks, edge–cloud computational asymmetry, and multi-objective conflicts spanning energy efficiency, electricity cost, and thermal safety. To address these challenges, this paper proposes an edge-cloud collaborative physics-informed reinforcement learning framework for production data center HVAC control. The framework integrates a physics-informed cold-start solution using Adaptive Particle Swarm Optimization (APSO) to generate physically constrained initial policies on a gray-box digital twin without expert demonstration data, a three-time-scale edge–cloud architecture coordinating minute-level edge Soft Actor-Critic (SAC) real-time inference, weekly edge APSO online model identification, daily cloud Non-dominated Sorting Genetic Algorithm III (NSGA-III) thermal storage scheduling, and a constraint-aware safe projection layer that embeds thermal safety hard constraints directly into the neural network policy. The framework is validated through a seven-month production deployment spanning the complete summer-to-winter transition, comprising approximately 3.2 million sensor records and evaluated with rigorous statistical methods.