Clement Z. Q. Ng

Gradient Descent with Stochastic Subspaces via Persistence of Memory

Stochastic subspace methods have gained popularity as gradient descent based techniques for large scale optimisation problems, especially in distributed settings. In this paper, we introduce the technique of"persistence of memory"to greatly extend and improve the random subspace methods. To this end, we leverage a vect...