Xiaodong Yang

The Gradient Descent Based Adaptive Optimizer Algorithm in Deep Learning

A lightweight short-term gradient-aware mechanism based on gradient sign consistency, which improves optimization stability while maintaining low computational complexity is proposed and two Adam-based optimizers are developed by incorporating adaptive friction coefficients constructed through Sigmoid and Tanh mappings...