Non-IID Data Challenges in Federated Learning for Distributed Systems: A Systematic Review

Abstract

: Federated learning is a distributed machine learning framework, that enhances privacy and security against malicious behavior, including data poisoning and model poisoning. Nevertheless, federated learning models tend to have worse performance than centralized models, particularly when the data is non-independent and identically distributed (non-IID) on local devices. Moreover, the non-IID data negatively impacts the outcome of the model, causing slower convergence, lower accuracy, and even leading to model bias. These challenges limit the applicability of the federated learning model and exhibit constrained convergence and generalization. In order to help researchers better understand and study the challenges of non-IID data settings in federated learning, this article presents a systematic review of existing research by analyzing beyond 50 related works and providing substantial solutions to address certain challenges in non-IID data. Unlike existing literature, this survey introduces a sophisticated, comprehensive taxonomy that systematically categorizes non-IID data challenges across statistical dimensions, critically evaluates mitigation strategies, and outlines open research trajectories for federated learning. The review discussed the taxonomy of the non-IID data challenges with their statistical properties, such as label, temporal, and quantity. Moreover, the review describes popular solutions to mitigate the challenges, ensure personalized federated learning and a domain adaptation technique in distributed and edge-based intelligent systems, including internet of things (IoT) and communication networks, and outlines promising future research directions.