This paper proposes a novel Largest Distance Client Selection (LDCS) method that prioritizes clients based on the divergence of their local models from the global model, as quantified by the Frobenius norm, thereby improving training efficiency and model performance while overcoming the limitations of existing random or loss-based approaches.
Federated learning (FL) is an emerging distributed machine learning framework that enables collaborative learning among multiple parties while preserving data privacy. However, the complexity of environments and node heterogeneity in the real world result in uneven data distribution across nodes, leading to Non-IID (Non-Independent and Identically Distributed) characteristics in data distribution. Such data distribution significantly reduces the convergence and performance of the model, becoming one of the fundamental challenges in federated learning mechanisms. To address the above issue, this paper proposes a novel FL framework-FedGKD. For the Non-IID client data distribution problem, we employ client-side local data augmentation, where GAN models are deployed on each client to generate synthetic samples so that local data distribution imbalance can be effectively alleviated. To further overcome the limitations of client-side local data augmentation under Non-IID, FedGKD introduces server-side privacy-preserving data-free knowledge distillation, which can transfer the knowledge of selected clients to the server while ensuring privacy protection, further mitigating the impact of Non-IID on federated learning and solving the problem of model performance degradation caused by direct aggregation. Extensive experiments demonstrate that FedGKD significantly outperforms the baseline algorithms in terms of accuracy, while exhibiting excellent performance in other metrics.
Furui Zhan, Ziyu Deng, Yingxin Liu et al.· IEEE Transactions on Informa...· 0 citations
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that"increasing the number of global models yields better initialization,"and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P) topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.
Mirko Konstantin, S. Zachow, Anirban Mukhopadhyay· 0 citations
A review of federated learning through a structured taxonomy that covers its core architectural paradigms, major learning types, model training approaches, and aggregation mechanisms, and analyzes the principal challenges confronting FL, including privacy and security risks, statistical and system heterogeneity, communication constraints, and global model divergence.
Mahdiyeh Velaei, Hosna Ghahramani, Ali Ghaffari et al.· Cluster Computing· 0 citations
The fast growth of big data, along with the growing stricter privacy laws, poses tremendous challenges to the traditional centralized implementation of AI models. Federated Learning (FL) offers a decentralized solution; however, it is undermined by a number of performance limitations such as large communicational overhead, statistical heterogeneity due to nonindependent and identically distributed data, and small computational capability of edge clients. This paper presents an optimized AI-based FL architecture, which consolidates a client selection algorithm (that is based on reinforcement-learning) and a dynamic and fairness-conscious aggregation protocol, as well as a new query-fragment caching solution. The combined model is strategically focusing on high utility players, reducing skew in data during model aggregation, and lessening on-device unnecessary processing. The empirical analysis of CIFAR-10 and FEMNIST shows that the presented framework reduces the communication rounds by 38% and the model accuracy is increased by 5.7% when compared with Fed Avg and Fed Prox, at the same time, data privacy is also rigorously maintained.
Shashi Purna Kurapathi· International Conference Com...· 0 citations
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Jiahao Wu, Ming Hu, Yanxin Yang et al.· Advances in Neural Informati...· 1 citation