Federated Reinforcement Learning (FedRL) improves sample efficiency while preserving privacy; however, most existing studies assume homogeneous agents or utilize public datasets for knowledge distillation to address agent heterogeneity, which limits its applicability in real-world heterogeneous scenarios. Knowledge distillation (KD) is an effective approach to facilitate knowledge sharing among heterogeneous models, but applying it to FedRL faces challenges such as the scarcity of public datasets and restricted knowledge representation. To address this, we propose a Federated Knowledge-Generating Distillation framework (FedKGD), which autonomously generates pseudo states through a learnable generator, completely eliminating reliance on real public datasets. The generator is optimized with a diversity-maximizing loss to ensure coverage of high-value state spaces; agents achieve efficient knowledge transfer among heterogeneous policy networks by uploading their policy distributions over the pseudo dataset. Theoretical analysis demonstrates that the method converges at a rate of $\mathcal{O}\left( {1/\sqrt T } \right)$. Extensive experiments on various reinforcement learning benchmark tasks show that the proposed framework significantly enhances learning stability and asymptotic performance, and maintains superior results even without relying on any public datasets. This provides a new paradigm for privacy-preserving distributed reinforcement learning.
Federated Continual Learning (FCL) enables distributed clients to collaboratively learn a sequence of tasks while preserving data privacy and mitigating catastrophic forgetting. However, most existing FCL methods rely on the assumption that all clients share an identical model architecture, which is impractical in real-world federated systems due to diverse client resources. Although logit-based federated distillation provides a feasible solution for model-heterogeneous collaboration, the quality of aggregated logits would be severely degraded under continual learning scenarios, where model heterogeneity, data heterogeneity, and accumulated previous classes jointly introduce biased and unstable teacher signals. To address these challenges, we propose FedDKD, a dual knowledge distillation framework for model-heterogeneous federated continual learning. FedDKD jointly exploits logits and prototypes as complementary knowledge carriers to integrate sample-level decision knowledge with class-level semantic knowledge. Specifically, prototype ensemble distillation further provides semantic anchors for global model to align feature spaces and reduce the bias of logits aggregation on public data. Furthermore, FedDKD decouples the distillation of current-class and previous-class knowledge to balance plasticity and stability during continual learning, and introduces an inter-task prototype transfer mechanism to generate pseudo feature prototypes of previous classes without accessing raw previous-task data. Extensive experiments show that FedDKD achieves competitive performance compared to mainstream methods.
Pei-Yi Zeng, Shu-Ming Yang, Jia-Wei Liao et al.· 2026 12th International Conf...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
