SHEFL: Sparse Heterogeneous Ensemble Federated Learning with Group-Balanced Aggregation
TL;DR
Across four vision datasets, SHEFL improves over Fed-Ensemble (FedEns) in all data heterogeneity settings and under a matched total uplink budget, SHEFL reaches the target accuracy in fewer rounds than FedEns and improves final accuracy by up to 14.65 points.
Abstract
System heterogeneity in federated learning (FL) requires training workloads and communication demands to be adapted to client resource capacities. Federated ensemble learning distributes multiple predictors across clients, but assigning one ensemble member to each client reduces the number of contributors to each member as the ensemble grows. We propose SHEFL, a global ensemble-based FL framework that uses the additional computation of high-power clients (HPCs) to train all ensemble members while low-power clients (LPCs) train one assigned member. SHEFL addresses increased HPC uplink demand and biased aggregation toward HPC updates through resource-aware sparse update allocation and group-balanced aggregation. Across four vision datasets, SHEFL improves over Fed-Ensemble (FedEns) in all data heterogeneity settings. Under a matched total uplink budget, SHEFL reaches the target accuracy in fewer rounds than FedEns and improves final accuracy by up to 14.65 points. Ablations further show that group-balanced aggregation improves accuracy and convergence under different resource allocations.
