Ethan Huffman

Long-Time Behavior of Zero-Noise Mean-Field Adam

Adaptive methods such as Adam are the most commonly used methods for training neural networks, but their long-time behavior is not as well understood as that of non-adaptive methods. This is particularly true in the mean-field limit, where the dynamics of infinitely many neurons are described as a distribution over par...