Estimating Execution Progress of Black-Box Tasks with Resource-Consumption Models

Abstract

In cloud environments, recurring workloads such as data processing jobs, CI/CD tasks and machine learning pipelines are typically executed as opaque black-box programs. Their internal progress is not directly observable, yet their resource-consumption patterns often exhibit structural similarity across runs despite variations in input data and hardware. Accurately estimating the progress of such running tasks can bring major benefits to scheduling and resource optimization. Using resource-consumption models of such tasks, derived from previous task execution monitoring, to accurately estimate a tasks cumulative progress allows detailed and accurate proactive resource steering, planning and scheduling. We present a novel approach that, given a generic resource-consumption model, estimates, based on current resource monitoring data, the progress of an ongoing execution in the form of a probability distribution which supports uncertainty quantification. The approach accounts for deviations from the model caused by varying system performance profiles, input sizes and parameters. It performs estimation via Markov-chain Monte-Carlo sampling around a maximum a posteriori estimate. Synthetic evaluations show that the approach reliably estimates task progress with an error of approximately 0.09 when suitable priors were selected. Real-world demonstrations show that the approach remains effective, and that performance degrades only slightly when the method is applied to real data. The degree of robustness to poor prior selection has also been thoroughly explored.