A momentum-based variance-reduced algorithm for federated multiobjective optimization that incorporates a momentum-driven gradient estimator into the local updates to reduce the variance of stochastic updates, leading to an improved convergence rate.
Abstract
Federated learning has traditionally been formulated as a single-objective optimization problem, primarily focused on maximizing model utility. In real-world applications, however, machine learning models often need to optimize multiple and potentially conflicting objectives simultaneously. This motivates federated multiobjective optimization (FMOO), which provides a natural framework for jointly handling multiple task-specific objectives in federated learning. In this paper, we propose a momentum-based variance-reduced algorithm for federated multiobjective optimization. The method incorporates a momentum-driven gradient estimator into the local updates to reduce the variance of stochastic updates, leading to an improved convergence rate. We establish theoretical guarantees showing that the expected Pareto stationarity measure of a randomly selected output iterate decays at a rate of $\mathcal{O}(T^{-2/3})$, improving upon the $\mathcal{O}(T^{-1/2})$ rates established for existing methods such as FSMGDA and FedCMOO. Numerical experiments on federated multiobjective optimization benchmarks demonstrate the effectiveness and competitive performance of the proposed algorithm.
This work establishes an efficient optimization framework for SFL under resource-constrained networks that jointly optimizes model splitting and resource allocation to minimize training cost, which is defined as the weighted sum of latency and energy costs.
Experiments on CIFAR-10, CIFAR-100, and AGNews under pathological and Dirichlet non-IID settings show FedFIbOS achieves higher accuracy than the state of the art, with improvements becoming more pronounced under stronger heterogeneity.
Yasmeen C. Afzal, Jeremiah D. Deng, Haibo Zhang· 0 citations
The groundwork for MOO is established, focusing on well-known approaches such as the weighted sum method alongside more advanced techniques like the multiobjective gradient descent algorithm (MGDA), and it is demonstrated that difficulties can arise regardless of the specific MOO method used.
Junaid Akhter, P. Fährmann, Konstantin Sonntag et al.· SIAM Review· 0 citations
Estimation error provably converges to zero as training progresses and HaFedHo surpasses state-of-the-art methods, including SCAFFOLD, MimeLite, and FedDyn, in both test accuracy and communication efficiency.
Jian-Rong Lu, Bang-Wei Li, Zhuo-Ya Gu et al.· Proceedings of the Thirty-Fi...· 0 citations
This work proposes an effective federated compositional Muon (FedCoMuon) optimizer to solve distributed matrix-wise compositional optimization problems and proposes a variance reduced variant of FedCoMuon (FedCoMuon-VR) based on a momentum-based variance reduced technique.
This paper departs from the stepwise MDP paradigm and reformulate AFL bidding as a Black-Box Optimization (BBO) problem, and proposes BBO-AFL (Budget-Efficient BBO for DC in AFL), a Budget-Aware Exploration mechanism that decouples high-frequency market cost signals from low-frequency model utility feedback.
Xiao-Li Tang, Haoran Shi, Han Yu et al.· Proceedings of the 32nd ACM...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.