A Carbon-Optimal and Security-Compliant Scheduler for Cloud Workloads using Region-Specific ML Models

Abstract

Data centers in the cloud use a lot of energy and produce considerable carbon emissions because of the growing requirements for computational and AI-heavy processes. Currently, available cloud schedulers pay attention to performance and resource usage optimization without much consideration of carbon footprint efficiency, security compliance, and region-based regulations for cloud data centers. In this research paper, we suggest creating a Secure Carbon-Aware Scheduler that includes machine learning-based carbon intensity prediction, prioritization of the workload, and compliance-based scheduling for distributed cloud environments. Our solution includes using a hybrid approach to carbon monitoring by integrating live carbon intensity data from one pilot region with predicted data based on machine learning models for distributed regions. We trained three ML algorithms such as Random Forest, Gradient Boosting, and Linear Regression for three years of data on carbon intensity, and the best models were chosen according to R2 score and MAE metric.The scheduler performs an assessment of potential regions through a multi-criteria objective scoring function that is based on carbon footprint, security compliance, workload prioritization, and system performance. The architecture employs security enforcement methods that support policies such as GDPR and HIPAA, as well as asymmetric cryptography-based security features for sensitive workloads. Experiments have been conducted by simulating cloud workloads from fourteen different regions and showed promising results in terms of workload prioritization and better carbon-aware scheduling when compared to conventional scheduling algorithms.