Particle swarm optimization (PSO) is a widely used algorithm featured in many state-of-the-art optimization tool-kits. However, rigorous performance guarantees are still lacking. The standard PSO dynamics do not admit a natural mean-field description, which would provide an avenue for theoretical analysis. By modifying the PSO formulation, one can recover the consensus-based optimization (CBO) algorithm with memory, which admits a mean-field limit and facilitates rigorous convergence analysis. These theoretical guarantees rely heavily on the fact that for CBO, the drift and noise strengths can be chosen independently, whereas they are coupled for PSO. We analyze how the PSO parameter coupling affects existing convergence guarantees for CBO and its variant with memory effect. We show, by an explicit construction, that the coupling still leaves a non-empty set of admissible parameters for these convergence guarantees to hold. However, the admissible parameter ranges shrink in the limits used to recover PSO. The resulting convergence guarantees from CBO therefore do not directly extend to the classical PSO model. We provide numerical simulations illustrating the parameter tradeoffs shown in the theoretical analysis.
Franca Hoffmann, Dohyeon Kim, Ritvik Teegavarapu· 0 citations
Particle Swarm Optimization (PSO) is a global optimization algorithm defined by an interacting set of particles evolving over the search space. Heuristically motivated, its theoretical analysis remains limited due to the second-order, stochastic, and highly nonlinear nature of the dynamics. In this paper, we connect classical PSO stability analysis under the stagnation assumption with more recent mean-field methods, providing new quantitative estimates for the time-discrete algorithm. We study in particular a regularized PSO model without memory, with non-degenerate noise by adding a noise floor to the original model. Studying such a surrogate model allows us to identify quantitative conditions under which the dynamics is stable and converges toward a small neighborhood of a global minimizer. We do so by first studying the Schur stability of the linearized dynamics, then analyzing the convergence properties of a nonlinear mean-field system via a Laplace principle, and finally establishing a quantitative error bound for the mean-field approximation of order $N^{-1/2}$.
We establish quantitative convergence to the target and uniform-in-time propagation of chaos for Langevin-regularized Stein variational gradient descent. The Stein interaction need not be small relative to the confining Langevin drift and does not generally yield a contractive particle coupling. At the mean-field level, the Stein and Langevin components dissipate the same relative entropy in the kernel-induced Stein and $2$-Wasserstein geometries, producing the squared kernel Stein discrepancy and relative Fisher information. Under a log-Sobolev inequality for the target, this yields exponential last-iterate convergence. We also derive a finite-particle entropy identity relative to the product target, giving exponential-in-time convergence of the empirical measure up to polynomial sampling errors. For propagation of chaos, we develop two complementary finite-time approaches. A synchronous coupling, combined with exponential moment estimates for the nonlinear mean-field diffusion, yields explicit single-exponential bounds in Wasserstein distance and kernel Stein discrepancy (KSD). Moving-product entropy gives joint-law relative entropy control relative to the evolving mean-field product law and, through entropy superadditivity and concentration, fixed-marginal relative entropy and total variation bounds and empirical KSD estimates. Under an additional $T_2$ inequality for the initial law, it also yields Wasserstein bounds. Combining these finite-time estimates with target convergence at a logarithmic cutoff time gives polynomial uniform-in-time propagation of chaos rates in expectation for empirical KSD and $W_2^2$, and for fixed-marginal total variation and $W_2^2$. All bounds control the last iterate in physical time. We also compare the two finite-time mechanisms and identify regimes in which each gives the sharper polynomial exponent.
Sayan Banerjee, Dohyeon Kim· 1 citation· ⚡1
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
