Context-Aware Dynamic Momentum for Asynchronous Federated Learning

Abstract

Asynchronous Federated Learning (AFL) addresses the synchronization bottleneck of traditional federated learning by allowing clients to upload local model updates independently. However, asynchronous communication can result in stale updates, whose optimization information can become outdated in cases of heterogeneous (Non-IID) data distribution. Momentumbased approaches typically have a fixed momentum coefficient, which implies that fresh and stale updates are both reliable. We propose Context-Aware Dynamic Momentum (CADM), a staleness-aware momentum adaptation system that dynamically adjusts the momentum coefficient based on update staleness using an exponential decay function. Preliminary were conducted under multiple combinations of data heterogeneity and network conditions, including momentum decay strategy evaluation and long-term training analysis. Experimental results demonstrate that CADM consistently outperforms the conventional fixedmomentum approach, achieving up to an 18.7 percentage-point improvement under severe Non-IID conditions while reducing the regression gap from 16.4 to 4.9 percentage points. These results indicate that adaptive momentum based on update staleness effectively improves optimization consistency and robustness in heterogeneous asynchronous federated learning environments.