HRL-TaskOpt: A Hierarchical Reinforcement Learning-Based Task Scheduling Framework for Multi-Cloud and Hybrid Environments

Abstract

Cloud computing has emerged as a new paradigm, which entrusts task scheduling to ensure the satisfaction of stringent constraints on latency, energy, and resources for sustainably running real-time applications. State-of-the-art natural DRL-based scheduling solutions mainly rely heavily on DRL techniques and are either limited in scalability, adaptivity, or generality of workloads/infrastructures. State-of-the-art flat methods, such as DQN and actor–critic models, are not sufficiently effective at high levels of decision complexity and are not robust against varying system loads and task priorities. In this paper, we present HRL-TaskOpt, a novel Hierarchical Reinforcement Learning-based task scheduling framework that combines high-level global task offloading with millisecond-granularity local scheduling policies in an edge–cloud scenario. In the proposed framework, there are two levels of agents: a high-level policy that utilizes Proximal Policy Optimization (PPO) to select the optimal execution tiers (edge or cloud), and a low-level policy based on Deep Q-Networks (DQN) to manage scheduling within nodes (edge or cloud). Such decomposition enables HRL-TaskOpt to efficiently accommodate heterogeneous workloads and adapt to dynamically changing infrastructure. We use synthetically generated workloads that reflect the characteristics of real-world applications to demonstrate the effectiveness of our model and compare it with state-of-the-art models such as SA-DQN, DRL-DO, and GD-DRL. We experimentally demonstrate that HRL-TaskOpt achieves a time reduction of up to 21.4% for task completion (2,000-task workload vs. SA-DQN), an energy efficiency improvement of up to 20.3% (10,000-task workload vs. GD-DRL), and a task success rate improvement of up to 13.3% (averaged across baselines at the 2,000-task workload), compared to these models. In addition, robustness to resource failures and sensitivity to task-type diversity, demonstrated in images, validate the real-life usability of the model. HRL-TaskOpt offers a scalable and intelligent solution for adaptive task scheduling, making it an appealing candidate for deployment in next-generation edge–cloud continuum systems.