Adaptive Hybrid PSO (AHPSO), which uses a sigmoid function on swarm diversity to automatically modulate gradient influence: near-zero during exploration, near-maximum during exploitation, with no manual phase-switching, is proposed.
Abstract
Gradient injection helps Particle Swarm Optimization (PSO) only when the swarm has identified a basin with smooth local structure, not universally. We propose Adaptive Hybrid PSO (AHPSO), which uses a sigmoid function on swarm diversity to automatically modulate gradient influence: near-zero during exploration, near-maximum during exploitation, with no manual phase-switching. Under budget-normalized comparison (PSO given equivalent total function evaluations), PSO wins 52.5% of 40 configurations versus AHPSO's 20% (p = 7.0e-5, Friedman). AHPSO retains advantage specifically on problems with smooth local basins (F8, F24-F27) where directed descent outperforms undirected sampling even at equal cost. Under iteration-matched comparison across 29 functions (42 configurations, 14,700 runs), AHPSO-Adadelta ranks first of 9 methods including CMA-ES (p = 9.75e-4). The contribution is a principled characterization of when gradient injection provides value in swarm-based search, not a claim of universal superiority.
Traditional particle swarm optimization (PSO) easily falls into premature convergence, while differential evolution (DE) is highly sensitive to fixed control parameters. Existing PSO-DE hybrid frameworks suffer from static population sizes and insufficient cross-population information exchange. This paper proposes PSO-DE-ADP, a hybrid optimizer with sinusoidal adaptive parameters, elite-guided mutation, ring neighborhood-weighted PSO and fitness-driven dynamic dual-population allocation. Four complementary mechanisms are integrated: (i) sine-wave perturbation superimposed on linear decay adaptively adjusts PSO inertia weight, acceleration factors and DE scaling/crossover coefficients to balance search stages; (ii) global elite individuals are embedded into DE mutation to reduce blind random search; (iii) ring topology with weighted learning realizes bidirectional information interaction between PSO and DE subpopulations; (iv) the proportion of PSO/DE individuals is dynamically adjusted according to elite ratio to allocate computing resources. Experiments adopt the CEC2017 30-dimensional benchmark with 30 test functions covering unimodal, multimodal, hybrid and composite landscapes. Compared with 8 state-of-the-art metaheuristics, PSO-DE-ADP achieves the lowest Friedman rank (1.08 vs. 2.23–4.90 for PSO variants; 1.53 vs. 2.07–5.00 for non-PSO algorithms). Ablation tests prove each component significantly boosts accuracy; The algorithm only costs 0.172 s average runtime, superior to all competitors. Statistical Wilcoxon and Friedman tests verify its significant superiority. Future work extends this method to multi-objective, constrained and real engineering optimization tasks.
Yao-Pei Wang, Yufeng Wang, Ke Liu· Algorithms· 0 citations
Particle swarm optimization (PSO) is a widely used bio-inspired optimization method. However, global guidance can align particle trajectories, reduce population diversity, and lead to premature convergence. To address this issue, we propose GRACE-PSO, a particle swarm optimizer based on group rank assessment and cooperative evolution. The method introduces group-best learning as an intermediate layer between personal-best and global-best learning to improve the balance between exploration and exploitation. GRACE-PSO integrates three coupled mechanisms: (1) a group-learning update that provides the population with multiple group-specific search directions; (2) a rank-based group-utility assessment that evaluates relative search effectiveness through pairwise comparisons of personal-best fitness values; and (3) a utility-driven adaptive strategy that adjusts the strengths of group-best and global-best learning and selectively reinitializes a small number of underperforming particles in stagnant groups. Experiments on 29 CEC 2017 benchmark functions at 30 and 50 dimensions against seven representative PSO methods show that GRACE-PSO achieves average ranks of 1.2414 and 1.3793, respectively. Experiments on two practical flexible intelligent metasurface optimization problems further demonstrate its competitive performance and practical applicability.
Particle swarm optimization (PSO) is a widely used algorithm featured in many state-of-the-art optimization tool-kits. However, rigorous performance guarantees are still lacking. The standard PSO dynamics do not admit a natural mean-field description, which would provide an avenue for theoretical analysis. By modifying the PSO formulation, one can recover the consensus-based optimization (CBO) algorithm with memory, which admits a mean-field limit and facilitates rigorous convergence analysis. These theoretical guarantees rely heavily on the fact that for CBO, the drift and noise strengths can be chosen independently, whereas they are coupled for PSO. We analyze how the PSO parameter coupling affects existing convergence guarantees for CBO and its variant with memory effect. We show, by an explicit construction, that the coupling still leaves a non-empty set of admissible parameters for these convergence guarantees to hold. However, the admissible parameter ranges shrink in the limits used to recover PSO. The resulting convergence guarantees from CBO therefore do not directly extend to the classical PSO model. We provide numerical simulations illustrating the parameter tradeoffs shown in the theoretical analysis.
Franca Hoffmann, Dohyeon Kim, Ritvik Teegavarapu· 0 citations
Particle Swarm Optimization (PSO) remains highly effective on many continuous optimization problems, yet its search dynamics collapse too quickly on high-dimensional and deceptive landscapes. Our earlier work introduced single-role diversity-enhancing PSO variants and showed that diversity injection is effective only when embedded in swarm dynamics in a structurally meaningful way. The present study extends those previously published single-role variant results. It introduces a unified role-based framework for informed diversity control. The main benefit, we show, comes not from isolated operators, but from composing behavioral roles across the social and cognitive velocity components. The framework organizes diversity mechanisms into three families: repulsion from best solutions, attraction toward worst solutions, and repulsion from worst solutions. Beyond single-role variants, we introduce paired-role formulations and three hybrid architectures allowing different roles to act on different velocity components within the same swarm. Empirical evaluation on 32 benchmark functions across dimensionalities up to 1000 reveals that these multi-hybrid strategies consistently outperform standard PSO and remain highly competitive with state-of-the-art algorithms such as CMA-ES and L-SHADE in high-dimensional spaces.
Piotr Urbańczyk, Aleksandra Urbanczyk, Leszek Rutkowski et al.· Vietnam Journal of Computer...· 0 citations
Comprehensive experiments on numerical benchmarks and neuroevolution robotic control tasks demonstrate that AutoPSO consistently discovers novel PSO variants that significantly outperform strong baselines and confirm that AutoPSO achieves increasing performance gains with larger swarm sizes.
Xin-Meng Yu, Jia-Xin Gao, Jianguo Zhang et al.· IEEE Transactions on Evoluti...· 1 citation
Gradient Descent (GD) is among the classic and most common optimization algorithms in machine learning, signal processing and numerical optimization. Although conceptually appealing and simple, traditional gradient descent with a constant learning rate is prone to slow convergence, oscillation or divergence, particularly in high-dimensional non-convex or ill-conditioned optimization problems. The choice of a suitable size of steps (learning rate) is also a critical issue since the progressively small step sizes results in very sluggish convergence, and conversely, the large step size can overshoot the minima. This paper describes a creation of a better gradient descent based on adaptive step-sizing, which varies the learning rate at each training step based on the gradient tendencies and previous update data. The suggested methodology will be designed to make convergence faster, achieve better numerical stability, and have more robustness in a broad longitude of optimization problems. We present a complete formulation of theory, the design of algorithms and their comparison with classical gradient descent and other popular adaptive versions. The presence of extensive experiments on benchmarks of optimization functions and machine learning tasks demonstrate that adaptive step-sizing has a substantial positive effect both on the rate of convergence and the quality of the solution and on the sensitivity to hyper parameter++ starting point. The findings prove that when used in contemporary optimization problems with large scale, adaptive gradient descent techniques are feasible and scaled to large scale problems. The contribution of this work is a framework that is well structured and can be easily incorporated into the existing learning systems and optimization pipelines.
D. Sharma, Vikram Sethi· International Journal of Mac...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
