TL;DR
FedALA-WR, a novel framework that integrates adaptive attention-based aggregation with a weighted memory replay mechanism, allowing clients to reinforce valuable past knowledge during local training, consistently outperforms baseline FL and state-of-the-art pFL methods in accuracy and convergence speed.
Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P) topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.
FedRAM is proposed, a three-step framework that progressively updates two scalar hyperparameters: the task importance weight and the client aggregation coefficient, where the proxy model serves as an intermediate between the local reference model and the global agent model.
Personalized Federated Learning (pFL) has emerged as a promising paradigm, while existing approaches face 3 limitations: granularity mismatch, resource waste, and conflict aggregation. This paper presents a Personalized Federated Learning with Low-rank Pruning-based Adaptation (pFedLoPA) framework. Instead of balancing global and local trade-offs, pFedLoPA decouples the model into client-specific cores and globally shared complements. It integrates low-rank adaptation to constrain optimization to a compact subspace, gradient-based pruning to identify personalized parameters, and complementary aggregation to exchange only relevant updates. This enables clients to retain critical knowledge locally while efficiently integrating global knowledge. Extensive experiments on CIFAR-10/100 with multiple network architectures demonstrate that pFedLoPA outperforms state-of-the-art methods in test accuracy (up to 94.31% on CIFAR-10) while reducing communication costs by over 70%.
A novel FL framework is presented, FedPhoenix, that stochastically re-sets partial parameters in each round to destroy some features of the global model, guiding FL training to learn multiple generalized features for inference rather than specific overfitting features.
Federated Learning often suffers from significant performance degradation under Non-IID (non-identically and independently distributed) data settings, where heterogeneous client data distributions lead to severe model drift. Traditional aggregation strategies, such as FedAvg, typically determine client contribution weights based solely on local dataset sizes, neglecting the varying quality and relevance of individual client updates to the global model. In this paper, we propose a novel Improvementbased Reliability Weighted Federated Learning (IRW-FL) framework. The key idea is to estimate client reliability by measuring the net performance improvement achieved on a small local validation set after each round of local training. To reduce the impact of stochastic fluctuations across training rounds, we maintain an Exponential Moving Average (EMA) of these improvements to construct a stable long-term reliability estimate for each client. Following a brief warm-up phase, the reliability scores are normalized using a temperature-scaled Softmax function and combined with client dataset sizes to produce adaptive aggregation weights. In addition, we incorporate serverside momentum to further stabilize the global optimization process. Extensive experiments on the CIFAR-10 dataset using a lightweight CNN model under multiple Dirichlet-based Non-IID scenarios demonstrate that IRW-FL consistently outperforms strong baselines, including FedAvg and FedProx. We further conduct ablation studies on warm-up duration and temperature parameter sensitivity, and evaluate robustness under partial client participation. The proposed approach achieves higher global model accuracy and exhibits strong robustness under severe data heterogeneity.
Results indicate that multi-factor reliability assessment and heterogeneity-aware adaptive forgetting improve the balance between useful knowledge retention and harmful-knowledge suppression in heterogeneous FCL.