Benchmarking Optimiser Choice and Privacy-Risk Dynamics in Non-IID Federated Learning

Abstract

Federated Learning (FL) trains shared models across distributed clients without pooling raw data, but its behaviour under heterogeneous data distributions and the associated privacy risks remain imperfectly understood. This paper presents a controlled empirical comparison of Federated Averaging (FedAvg), Federated Proximal (FedProx), Federated Adaptive Moment Estimation (FedAdam), and Differentially Private Federated Averaging (DP-FedAvg) under matched conditions. The experiments use CIFAR-10 and MNIST with Non-Independent and Identically Distributed (non-IID) client partitions, including Dirichlet and class-restricted partitioning. Each condition is repeated across 20 independent seeds. We examine how optimisation choice shapes convergence and stability, how differential privacy affects classification performance, and how privacy exposure changes over the course of training. We find that FedAdam improves performance under severe client heterogeneity, where client gradients conflict strongly. FedProx changes the training trajectory, but we do not find it to improve cross-seed stability. DP-FedAvg reduces the success of gradient-reconstruction and membership-inference attacks, but this protection comes with a measurable reduction in classification performance due to gradient clipping and additive noise. A key finding is that privacy exposure is temporally concentrated rather than uniform across training. It is highest in the earliest rounds, when gradients are largest and most informative, and declines as training progresses. These results suggest that private FL systems should account for training dynamics, with particular emphasis on protection during early training rounds.