Privacy-Preserving Consensus-Oriented Aggregation for Federated Learning With Heterogeneity

Abstract

Achieving reliable network-wide consensus formation in distributed learning systems becomes increasingly challenging when edge nodes hold skewed data distributions. Federated learning (FL) enables privacy-preserving collaborative model training without sharing raw data, but statistical heterogeneity across nodes significantly degrades convergence and may expose sensitive label statistics to distribution-inference attacks. To address these limitations, this paper presents a new federated consensus-oriented aggregation (FedCOA) strategy, which improves consensus formation of FL under strong heterogeneity, suppresses skew-induced instability while mitigating distribution-level privacy leakage. FedCOA perturbs local label distributions using differential privacy (DP) and computes a noise-robust Index of Data Heterogeneity (IDH), which guides dynamic aggregation, regulates bias propagation, suppresses the influence of skewed updates, and facilitates consensus formation dynamics. We show theoretically that FedCOA reduces the divergence term in the convergence upper bound. Experiments demonstrate up to 80.3%, 75.1%, and 79.2% reductions in communication rounds on MNIST, FashionMNIST, and CIFAR-10, respectively.