IRW-FL: Improvement-Based Reliability Weighted Federated Learning

Abstract

Federated Learning often suffers from significant performance degradation under Non-IID (non-identically and independently distributed) data settings, where heterogeneous client data distributions lead to severe model drift. Traditional aggregation strategies, such as FedAvg, typically determine client contribution weights based solely on local dataset sizes, neglecting the varying quality and relevance of individual client updates to the global model. In this paper, we propose a novel Improvementbased Reliability Weighted Federated Learning (IRW-FL) framework. The key idea is to estimate client reliability by measuring the net performance improvement achieved on a small local validation set after each round of local training. To reduce the impact of stochastic fluctuations across training rounds, we maintain an Exponential Moving Average (EMA) of these improvements to construct a stable long-term reliability estimate for each client. Following a brief warm-up phase, the reliability scores are normalized using a temperature-scaled Softmax function and combined with client dataset sizes to produce adaptive aggregation weights. In addition, we incorporate serverside momentum to further stabilize the global optimization process. Extensive experiments on the CIFAR-10 dataset using a lightweight CNN model under multiple Dirichlet-based Non-IID scenarios demonstrate that IRW-FL consistently outperforms strong baselines, including FedAvg and FedProx. We further conduct ablation studies on warm-up duration and temperature parameter sensitivity, and evaluate robustness under partial client participation. The proposed approach achieves higher global model accuracy and exhibits strong robustness under severe data heterogeneity.