Muon and related optimizers update a weight matrix with a spectrally transformed gradient. In this work, we study such spectral optimizers with a constant step size on strongly convex matrix quadratics, which isolate the effect of the update from the confounds of nonconvexity, noise, and step size schedules. We analyze...
P. Jawanpuria, Bhabatosh Garai, Bamdev Mishra· 1 citation
The choice of Riemannian metric can strongly influence the convergence of gradient-based optimization over covariance matrices. Euclidean, Bures-Wasserstein and affine-invariant metrics are common choices, but their relative effectiveness depends on the objective. We introduce a two-parameter family defined by $X^{p}LX...
Yi-Bang Li, Bamdev Mishra, P. Jawanpuria et al.· 0 citations
This work extends the recently introduced Alpha-Procrustes family of Riemannian metrics for symmetric positive definite (SPD) matrices by incorporating generalized versions of the Bures-Wasserstein (GBW), Log-Euclidean, and Wasserstein distances. While the Alpha-Procrustes framework has unified many classical metrics i...
Salvish Goomanee, Andi Han, P. Jawanpuria et al.· arXiv.org· 0 citations
This work proposes a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product and develops a Riemannian gradient descent algorithm that uses a tuning-free Gaussian step size and scales linearly in the number of observed entries per iteration.