Sustainable Resource Management Framework for Hybrid Cloud Data Centers Using Explainable Machine Learning
Abstract
As clouds go hybrid, the challenge for hybrid cloud data centers is to optimize the use of cloud resources, reduce energy usage, and ensure adherence to service level agreements (SLAs). Machine learning based approaches for resource management till now are considered as black-box in which the operator cannot trust and cannot use it in production environments. In this paper, the authors present a Sustainable Resource Management Framework (SRMF) for hybrid cloud data centers that leverages Explainable Machine Learning (XAI) techniques: XGBoost, Random Forest, SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME) to offer transparent, interpretable, and energy-efficient resource allocation decisions. There is large-scale production data set of 8.3 million records over 30 days from Alibaba Cluster Trace v2021, Azure Public Dataset V1; and a smaller data set for test of 2.1 million records from Google Cloud Platform (GCP) Workload Traces 2023, also over 7 days. Infrastructure operators can gain insights from the dominant feature that influences their decision on resource allocation, which is CPU_Load_Avg $(\varphi=\text{0. 4 1 2) }$, followed by Memory_Usage $(\varphi=0.278)$ and Workload_ArrivalRate $(\varphi=0.148)$. Experimental results indicate that the proposed XGBoost+SHAP model outperforms the baseline linear regression in terms of prediction accuracy, achieving a value of 94.7% with RMSE=4.31, which is 27.5% higher than the baseline model' prediction accuracy of 74.3% with RMSE=12.84. Using the framework, the total amount of energy saved is 21.2%, SLA violations reduced from 18.4% to 1.9%, and CPU utilization increased from 52.1% to 84.2% over the static provisioning baselines. By helping to power cloud-based services with lower carbon footprint, foster intelligent infrastructure, improve sustainable urban digital services, and respective research optimize all data center resources, the SRMF is aligned with UN Sustainable Development Goals (SDG 7, SDG 9, SDG 11, SDG 13).