Momentum-Buffered Local SGD for Zero-Payload Drift Correction in Federated Learning

Abstract

Local SGD (FedAvg) reduces federated communication by running multiple local steps between server aggregations, but it suffers from “client drift” under non-IID data. Methods such as SCAFFOLD mitigate this drift by transmitting control variates, which roughly doubles the per-round payload. We propose Momentum-Buffered Local SGD (MB-LSGD), an algorithm that corrects client drift with no additional payload: each client maintains a local drift buffer updated from the standard model aggregate, and a half-step bias present in naive drift estimators is eliminated by an additive update rule. For smooth non-convex objectives, MB-LSGD attains an \(\mathcal {O}(1/\sqrt {M K R})\) convergence rate, matching centralized mini-batch SGD up to constants. The bound explicitly couples the buffer rate α with the participation rate p, providing concrete guidance for hyperparameter selection under partial client sampling. With 100 clients under Dir(0.1) partitioning, MB-LSGD reaches a peak accuracy of \(48.37\%\) on TinyImageNet, exceeding SCAFFOLD (\(47.49\%\)) at half the bandwidth, and a final accuracy of \(61.67\%\) on CIFAR-10, the highest among all stable baselines. MB-LSGD is also the only method whose final accuracy on CIFAR-10 matches its peak, indicating sustained convergence without late-stage degradation, and its halved per-round payload relative to SCAFFOLD translates to close to a 2 × reduction in end-to-end completion time in communication-bound deployments.