A Novel Fractional-Order Accelerated Gradient Descent Method for Nonlinear Optimization with Application to Posture Recognition
TL;DR
A Caputo fractional accelerated gradient descent method for unconstrained optimization problems that is applicable to both smooth and a class of non-smooth objective functions, and can achieve faster and more stable performance than competing approaches.
Abstract
This article proposes a Caputo fractional accelerated gradient descent (CFAGD) method for unconstrained optimization problems that is applicable to both smooth and a class of non-smooth objective functions. The proposed approach incorporates an adaptive (\b{eta}k)-parameter, which is heuristically updated throughout the iterative process to improve the search direction. Furthermore, the method employs the Caputo fractional derivative together with the adaptive (\b{eta}^k)-parameter, thereby preserving the memory characteristics associated with non-integer-order derivatives. A suitable step-size is selected via an inexact line-search technique based on the Armijo condition. The central idea is to scale the step-size by a positive parameter to improve the behavior of the iterates as they approach an optimal point, thereby generating a descent sequence. Under strong convexity and bounded Hessian assumptions, linear convergence of the proposed method is established. Numerical validations, including neural-network-based examples, further indicate that the CFAGD method can achieve faster and more stable performance than competing approaches.