I. Kunyo

Dynamic Regime Maps of Neural Network Training Under the Adam Optimizer: An Observable-Based Empirical Analysis

Adaptive optimization algorithms are fundamental to modern deep learning; however, the global organization of neural-network training regimes induced by optimizer hyperparameters remains insufficiently understood. In particular, the influence of the Adam moment coefficients on the stability and qualitative behavior...