FLEX: Flexible Federated Learning With Asynchronous Peer-to-Peer Communication and Adaptive Aggregation

Abstract

In this paper, we introduce FLEX, a flexible federated learning (FL) scheme that combines peer-to-peer communication with asynchronous client updates, enabling clients to operate independently. We further propose a novel aggregation algorithm inspired by gradient-like updates, governed by two key parameters: (i) a similarity-based aggregation weight to better handle non-IID data distributions, and (ii) a progress-based aggregation weight to mitigate straggler effects. Experiments on CIFAR-10, CIFAR-100, FE-MNIST, and Boston Housing Dataset demonstrate that our aggregation algorithm achieves accuracy improvements of up to 13% over simple averaging (Avg) and 10% over FedProx under both homogeneous and heterogeneous settings, as well as IID and non-IID data distributions. Beyond aggregation, the FLEX scheme substantially outperforms state-of-the-art asynchronous centralized schemes (TimelyFL, FedBuff), delivering over 16% higher accuracy while reducing communication costs by factors of 10.4 and 13.6, respectively, for the same target accuracy. Compared to synchronous peer-to-peer approaches (Epidemic), FLEX reduces total training time by 2–<inline-formula><tex-math notation="LaTeX">$3\times$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>3</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="sad-ieq1-3735216.gif"/></alternatives></inline-formula> and communication by 40–50%. Compared to asynchronous P2P baselines (AsyDFL), FLEX reaches target accuracy levels <inline-formula><tex-math notation="LaTeX">$4\times$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>4</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="sad-ieq2-3735216.gif"/></alternatives></inline-formula>–<inline-formula><tex-math notation="LaTeX">$7.2\times$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>7</mml:mn><mml:mo>.</mml:mo><mml:mn>2</mml:mn><mml:mo>×</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="sad-ieq3-3735216.gif"/></alternatives></inline-formula> faster while reducing the number of exchanged updates by 50%–71%. Finally, relative to traditional synchronous centralized FedAvg, FLEX achieves accuracy gains of up to 23%.