Preprint
Jul 2026
PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer
PoLoRA is introduced, a Preconditioned Orthogonalized LoRA optimizer built from three ingredients: a product-aware spectral update direction, curvature preconditioning derived from controlling the per-sample loss change, and a magnitude rule that controls the sizes of both the factor and merged updates.
Nikhil Ghosh, Tetiana Parshakova, Robert M. Gower
· 1 citation