Multi-Timescale Workload-Thermal Coordinated Control for AI Data Centers via Offline Multi-Agent Reinforcement Learning

Abstract

In AI data centers, IT equipment and cooling systems are the dominant energy consumers, jointly accounting for approximately 90% of the total energy footprint. While workload-thermal coordinated control offers energy-saving potential, effectively integrating these subsystems is challenged by the timescale discrepancy between rapid workload-side heat generation and slow thermal dynamics. Existing multi-timescale coordination architectures often decouple agents in ways that break Markov decision process (MDP) stationarity, hindering stable data-driven policy learning. To overcome this non-stationarity while minimizing energy consumption under server thermal constraints, we model joint workload allocation and air handling units regulation as a multi-timescale, cooperative multi-agent Dec-POMDP. We propose an offline centralized training with decentralized execution (CTDE) framework based on Implicit Q-learning (IQL). This framework incorporates a multi-timescale experience replay mechanism to safely derive control policies from historical data. Validated via an EnergyPlus-based simulator using real-world traces, our approach ensures a zero thermal violation rate, decreases cooling energy consumption by 19.4%, and improves power usage effectiveness (PUE) from 1.448 to 1.353 compared to conventional baselines.