This paper investigates stochastic multi-level optimization where the objective is a nested composition of several smooth non-convex functions. We assume that only stochastic estimates of the gradient and function values for each level are accessible. Consequently, obtaining an accurate estimate of the overall gradient...
Wei Jiang, Rui Yan, Si-Fan Yang et al.· 0 citations
Agnostic federated learning (AFL) seeks a model that performs reliably across $m$ heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the number of synchronization rounds via faster convergence and the communication cost per round via compression. We first prop...
Hao-Min Bai, Jun-Yan Sun, Si-Fan Yang et al.· 0 citations
In non-stationary online learning, dynamic regret has attracted increasing attention as a measure of how well an online learner performs against a time-varying comparator sequence. Despite considerable advances, attaining optimal bounds for strongly convex and exp-concave losses often involves intricate analysis. In th...
Yi-Bo Wang, Wen-Hao Yang, Si-Fan Yang et al.· 0 citations
This paper solves the problem of sign-based variance reduction methods' failure to obtain optimal convergence rates for both nonconvex stochastic and finite-sum optimization, and proposes tracking the global gradient at the server through unbiased compression of recursive gradient increments.
This paper studies projection-free algorithms for stochastic constrained multi-level compositional optimization and proposes variance-reduced projection-free algorithms and establishes complexity guarantees under both the Frank-Wolfe gap and the gradient mapping criteria.
Wei Jiang, Si-Fan Yang, Wen-Hao Yang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.