Privacy and Security in Knowledge Distillation for Federated Learning: A Survey

Abstract

Knowledge distillation (KD) is increasingly used in federated learning (FL) because it enables clients to exchange predictions, features, prototypes, or synthetic knowledge rather than full model parameters. This change can reduce communication and support heterogeneous models, but it also changes the privacy and security threat surface rather than eliminating it. This survey examines KD-based FL through a privacy and security lens. We distinguish formal differential-privacy guarantees, protocol-level cryptographic mechanisms, structural reductions of the attack surface, and empirical attack-specific defenses, and we map these notions to transfer direction, shared representation, and data-availability assumptions. We review attacks against released distilled knowledge, privacy-preserving distillation mechanisms, secure aggregation and encrypted-domain approaches, adversarial defenses, application domains, and open challenges for formal assurance and standardized benchmarking. The goal is to provide a precise framework for evaluating what KD-FL methods actually guarantee and what they only make harder for an adversary.