MOON: Multi-Objective OrthoNormalized Updates for Multitask Learning
This paper proposes MOON (Multi-Objective OrthoNormalized Updates), which performs gradient manipulation under spectral--nuclear norm geometry and uses the orthonormalized manipulated gradient for parameter updates and empirical results show that MOON consistently improves both optimization efficiency and final multi-t...