We have 2 of 12 papers
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
iKFAD is a recently proposed optimiser that replaces adaptive learning rates with adaptive friction in the momentum dynamics, yet performs as well as Adam. Its limitation is that the full friction tensor $\xi\in\mathbb{R}^{m\times n}$ carries the same $\mathcal{O}(mn)$ memory overhead per layer as Adam's second-moment...
Theoretical analysis demonstrates that the proposed estimator is unbiased, attains finite variance, and satisfies a central limit theorem, and the results demonstrate that in large-scale applications, the unbiased algorithm can be 2–3 orders of magnitude more efficient than the “gold-standard” randomized Hamiltonian Mo...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
