Non-Negative Generalized Parameter of the Hybrid Memoryless Three-Term Conjugate Gradient Method

Abstract

The Conjugate Gradient (CG) method is a well-established technique for solving optimization problems, particularly in large-scale applications. Despite its popularity, classical CG methods often face computational inefficiencies in high-dimensional tasks. This study addresses these limitations by developing the Hybrid Memoryless Three-Term Conjugate Gradient (HMTTCG) method, which introduces a non-negative generalized parameter to combine the strengths of several CG methods while reducing their weaknesses. Existing CG methods are classified into two groups: FletcherReeves (FR), Dai-Yuan (DY), and Conjugate Descent (CD) in the β1 group, and Liu-Storey (LS), Hestenes-Stiefel (HS), and Polak-Ribiere-Polyak (PRP) in the β2 group. These methods often suffer inefficiencies due to constraints like the non-negativity of CG parameters, which can disrupt sufficient descent conditions and lead to reliance on the steepest descent direction, increasing computational costs. To overcome these challenges, the HMTTCG method proposes a hybrid parameter that maintains descent properties and enhances computational efficiency by selecting suitable CG parameters. The approach involves deriving the hybrid parameter mathematically and conducting numerical experiments across various optimization problems, comparing the HMTTCG method with existing the three-term CG methods in terms of the number of iterations (NOI) and CPU time. The method ensures descending and bounded search directions, guaranteeing global convergence under the Strong-Wolfe line search. Experimental results demonstrate that HMTTCG outperforms existing methods by solving problems with fewer iterations and computational time, proving its efficiency and robustness. In conclusion, the HMTTCG method represents a robust and efficient optimization strategy for large-scale unconstrained optimization problems.