Bounding Queue Waits for Fair Scheduling of Bursty HPC Workloads via Hierarchical Algorithms
HPC clusters are shared among many projects, each holding an allocation of node-hours, and ensuring fairness is becoming harder. Foundation models workflows and automated pipelines have made job submissions bursty and concentrated among a few heavy users, so occasional users absorb queueing delays created by frequent o...