Straggler-Aware Asynchronous Federated Learning via Temporal Cost Clustering for Fair and Efficient Heterogeneous Model Training

Abstract

Heterogeneous federated learning leads to system and data differentials that cause stragglers to either be a bottleneck to synchronous optimization or create representation bias in asynchronous contexts. Although current approaches deal with staleness or buffering independently, their approach does not ensure fast clients do not take over the global model. The proposed framework Straggler-Aware Asynchronous Federated Learning (SAFL), that re-defines the stragglers as structured subjects rather than outliers. SAFL employs temporal exponentially weighted moving average signature of client costs and costs model updates by clustering costs in time-constrained per-cluster buffers. An innovative fairness-sensitive aggregation scheme then balances the participation through frequency compensation and damping on staleness. The results of the experiment indicate that SAFL achieves a 75% accuracy in 620 seconds, 27% higher than the state-of-the-art Federated Asynchronous Mobile Update (FedASMU) and increases the fairness index by 0.52 to 0.87. SAFL has a scalable, fair approach to the regulation of heterogeneous clusters, which means they can be used to ensure almost equal contribution in regulated settings such as financial and healthcare analytics.