Load Balancing with Individually Heterogeneous Servers

Abstract

In this paper, we analyze the diffusion limit of Join-the-Shortest-Queue (JSQ) load balancing policies for a system with many parallel queues where each queue is being served by a server with a potentially different service rate. Prior asymptotic analyses of JSQ policies assumed servers to have service rates from a finite set which does not capture scenarios in modern data centers where service rates can vary at the level of individual servers. For systems with individually heterogeneous servers, tracking the empirical queue length distribution for each possible service rate becomes infeasible. To overcome this difficulty, we develop a framework based on measure-valued processes and provide a unified analysis of all JSQ-based load balancing policies under general tie-breaking rules. Our analysis identifies two key objects that distinguish the performance of different tie-breaking rules in the Halfin--Whitt regime, namely, the limiting fairness process which describes how idle servers are distributed across different service rates and the limiting routing measure which describes how arriving jobs are assigned across different service rates. In addition to characterizing the diffusion limits for different tie-breaking rules, we identify the tie-breaking rule which asymptotically minimizes the steady-state distributions of the diffusion-scaled total number of jobs and the diffusion-scaled number of waiting jobs. In proving these results, we develop crucial coupling-based sample-path comparisons which provide both policy-independent steady-state bounds and lower bounds to prove asymptotic optimality.