FedEMoE: Towards Class-Aware Federated Learning via Evidential Mixture-of-Experts

Abstract

Federated learning under non-IID data suffers from client drift. Recent work introduces Mixture-of-Experts (MoE) architectures to handle data heterogeneity by routing inputs to specialized experts, but two problems remain. Softmax routing produces overconfident expert assignments with no reliability measure. The parameter space carries no class-level information, forcing the server to aggregate all clients uniformly and destroying the expert specialization built during local training. We call this class-level specialization invisibility. To address these problems, we propose FedEMoE. On the client side, Evidential Mixture-of-Experts equips router and experts with evidential heads, dynamically adjusts the number of active experts based on routing uncertainty, and compresses accumulated evidence into a per-client signature. On the server side, Evidence-Guided Class-Aware Aggregation takes the signatures as input and adaptively performs class-aware parameter fusion and component recombination. Experiments on CIFAR-10, CIFAR-100, and FEMNIST show that FedEMoE consistently outperforms eight baseline methods across all settings. Our code is available at https://github.com/Stephen-Chow1/FedEMoE-CIKM.