Aug 2026· Applied Sciences· 0 citations· 11 references
TL;DR
Results indicate that multi-factor reliability assessment and heterogeneity-aware adaptive forgetting improve the balance between useful knowledge retention and harmful-knowledge suppression in heterogeneous FCL.
Abstract
Federated continual learning (FCL) must preserve useful historical knowledge while learning from evolving and statistically heterogeneous client streams. However, indiscriminate replay can retain client-specific bias, noise, or task conflicts and thereby cause negative transfer. In this paper, we propose Adaptive Multi-Factor Accurate Forgetting for Heterogeneous Federated Continual Learning (AMAF-FCL), a selective memory-management framework that jointly assesses replay reliability and adapts the influence of generated historical features. AMAF-FCL achieves this goal by (1) modeling historical knowledge in feature space with a conditional real-valued non-volume-preserving (RealNVP) normalizing flow; (2) combining class-conditional likelihood, predictive uncertainty, and global distribution consistency; and (3) adjusting replay weights according to client–global heterogeneity. The local objective combines a classification loss for learning the current task, a reliability-weighted replay loss for retaining useful historical knowledge, and a feature-distillation loss for limiting drift in the feature representation. In the EMNIST long-task-pool (EMNIST-LTP) benchmark, each client learns six two-class tasks drawn from a client-specific set of handwritten letters. AMAF-FCL achieves 51.2% average accuracy and 7.6% average forgetting, improving over the likelihood-based AF-FCL baseline by 3.7 and 1.5 percentage points, respectively. In the cross-domain digit-and-fashion setting, it reports 71.2% average accuracy and 6.8% average forgetting; with four noisy clients, it obtains 56.1% average accuracy and 9.8% average forgetting. These results indicate that multi-factor reliability assessment and heterogeneity-aware adaptive forgetting improve the balance between useful knowledge retention and harmful-knowledge suppression in heterogeneous FCL.
Personalized Federated Continual Learning (PFCL) requires that the server not only effectively integrate temporal knowledge accumulated across past tasks and spatial knowledge distributed among heterogeneous clients, but also ensure strong personalized performance of the global model for each client. Existing methods, whether in Personalized Federated Learning (PFL) or Federated Continual Learning (FCL), have overlooked the multi-granularity representation of knowledge, which can be utilized to overcome Spatial-Temporal Catastrophic Forgetting (STCF) and enable coarse-to-fine personalization. Furthermore, most approaches rely on local client-side personalization, increasing computational load and failing to address the risks posed by malicious or low-quality clients. To this end, we propose FedMGP+, which utilizes multi-granularity prompts to address these challenges, namely coarse-grained global prompt and fine-grained local prompt. The former focuses on efficiently transferring shared global knowledge without spatial forgetting, and the latter emphasizes specific learning of personalized local knowledge to overcome temporal forgetting. Visualization results and theoretical analyses further reveal that coarse-grained prompts primarily guide regional attention, whereas fine-grained prompts enrich object-level representations within those regions. Building upon this, Personalized Selective Prompt Fusion is designed to exclusively fuse coarse-grained knowledge on the server to generate client-specific global prompts, reducing local overhead and resisting poisoning attacks from malicious clients. Extensive experiments demonstrate that the proposed FedMGP+ effectively mitigates forgetting, significantly enhances personalized performance, and provides robust defense against poisoning and gradient leakage attacks.
Hao Yu, Xin Yang, Boyang Fan et al.· IEEE Transactions on Pattern...· 0 citations
Federated Learning often suffers from significant performance degradation under Non-IID (non-identically and independently distributed) data settings, where heterogeneous client data distributions lead to severe model drift. Traditional aggregation strategies, such as FedAvg, typically determine client contribution weights based solely on local dataset sizes, neglecting the varying quality and relevance of individual client updates to the global model. In this paper, we propose a novel Improvementbased Reliability Weighted Federated Learning (IRW-FL) framework. The key idea is to estimate client reliability by measuring the net performance improvement achieved on a small local validation set after each round of local training. To reduce the impact of stochastic fluctuations across training rounds, we maintain an Exponential Moving Average (EMA) of these improvements to construct a stable long-term reliability estimate for each client. Following a brief warm-up phase, the reliability scores are normalized using a temperature-scaled Softmax function and combined with client dataset sizes to produce adaptive aggregation weights. In addition, we incorporate serverside momentum to further stabilize the global optimization process. Extensive experiments on the CIFAR-10 dataset using a lightweight CNN model under multiple Dirichlet-based Non-IID scenarios demonstrate that IRW-FL consistently outperforms strong baselines, including FedAvg and FedProx. We further conduct ablation studies on warm-up duration and temperature parameter sensitivity, and evaluate robustness under partial client participation. The proposed approach achieves higher global model accuracy and exhibits strong robustness under severe data heterogeneity.
Zehua Li, Junnan Yang· 2026 6th International Confe...· 0 citations
Federated Continual Learning (FCL) is fundamental to real-world distributed learning systems, requiring models to adapt to sequential, non-IID data across clients while mitigating catastrophic forgetting and client drift. Existing approaches formulate continual learning (CL) as a sequence of per-task optimization problems, applied locally at each client and coupled through aggregation, using heuristic mechanisms such as replay, regularization, or projection-based constraints. However, forgetting in FCL is inherently a long-term, distributed phenomenon, arising from the interaction of temporal task evolution and cross-client heterogeneity, which is not explicitly regulated. In this work, we cast FCL as a stochastic control problem and propose Federated Queue-regulated Continual Learning (FedQCL), a framework based on Lyapunov drift-plus-penalty (DPP) optimization. FedQCL introduces virtual queues to track the accumulation of forgetting across tasks and clients, enabling explicit control of the stability-plasticity trade-off. By optimizing a DPP objective, the method jointly improves current-task performance while the queue-based formulation provides an interpretable and tunable mechanism to balance adaptation and retention through a single parameter, without requiring gradient projection or additional communication overhead. Empirical evaluations on standard benchmarks, including Split-CIFAR-10, Split-CIFAR-100, and Split-TinyImageNet, demonstrate that FedQCL outperforms state-of-the-art baselines with respect to accuracy while significantly reducing forgetting under heterogeneous data distributions.
Nazreen Shah, Naveen Kumar, Reddy Somireddy et al.· 0 citations
FedADB, a Class Anchor-Driven Dual-Branch FL framework, a dual-branch collaborative training mechanism designed for clients that achieves significant improvements in both accuracy and convergence speed.
Zhenyan Liu, Hua Zhang, Haoran Gao et al.· 0 citations
Federated fine-tuning of Multimodal Large Language Models (MLLMs) across distributed networks enables privacy-sensitive adaptation to evolving data streams, yet a fundamental obstacle prevents robust deployment in dynamic environments: catastrophic forgetting, wherein sequential task updates erase previously acquired knowledge across visual, linguistic, and cross-modal representations. Addressing this challenge is especially critical for autonomous networked AI operating in safety-sensitive domains, such as content moderation, where reliable retention of prior knowledge underpins system integrity. To overcome this, we propose Federated Continual Multimodal Learning (FedCMM), a framework that embeds continual-learning safeguards into the federated optimization loop at three complementary levels. At the parameter level, modality-aware elastic weight consolidation computes separate Fisher information matrices for the vision encoder, language backbone, and cross-modal projector, providing granular, asymmetry-aware protection against modality-specific forgetting. At the data level, each client trains a lightweight local generative replay module to synthesize raw-data-free embedding-level multimodal replay tuples without any raw data sharing. At the aggregation level, Task-similarity-aware gradient aggregation autonomously filters and reweights client updates by gradient cosine similarity, suppressing conflicting directions and stabilizing the global learning trajectory. Extensive experiments on two benchmarks demonstrate that FedCMM consistently outperforms recent baselines on accuracy and backward transfer, confirming that holistic, modality-aware optimization enables robust evolutive adaptation across heterogeneous networked AI deployments.
Jing Liu, Chenxuanyi Zou, Jiayang Ren et al.· 0 citations
FedRAM is proposed, a three-step framework that progressively updates two scalar hyperparameters: the task importance weight and the client aggregation coefficient, where the proxy model serves as an intermediate between the local reference model and the global agent model.
Fan Wu, Xinyu Yan, Jiabei Liu et al.· Neural Information Processin...· 0 citations