Subspace Levenberg Marquardt Algorithms in Training Neural Networks
This work evaluates the subspace Levenberg-Marquardt algorithms for regression and classification tasks in neural networks and compares the performance of subspace LM variants with the classical LM method, as well as other popular first-order algorithms, such as stochastic gradient descent and Adam.