LoRA-TSD: Tangent-Space Spectral Descent for LoRA via Muon-Style Updates
LoRA-TSD, an optimizer that treats every LoRA step as a tangent vector of the fixed-rank matrix manifold and takes the spectral-norm steepest-descent step of Muon inside that tangent space, mapping the result back to the factors through a retraction native to the LoRA parametrization, is introduced.