The Gradient Descent Based Adaptive Optimizer Algorithm in Deep Learning
A lightweight short-term gradient-aware mechanism based on gradient sign consistency, which improves optimization stability while maintaining low computational complexity is proposed and two Adam-based optimizers are developed by incorporating adaptive friction coefficients constructed through Sigmoid and Tanh mappings...