Optimizer Geometry Sets the Pace: Spectral Learning Dynamics in Matrix Factorization
Recent successes of matrix- and curvature-based optimizers have renewed interest in how update geometry shapes learning. These methods normalize or precondition updates, changing how different components progress during training. In deep matrix factorization, the geometry that slows gradient descent (GD) favors low-ran...
Mahalakshmi Sabanayagam, Simon Lucey
· 0 citations