Gradient-Momentum Coupling: A Parameter-Space Proxy for Learning Progress
Gradient-Momentum Coupling (GMC) is proposed, which measures how strongly a sample drives change in the parameter space, given by the normalized absolute product of its gradient with the momentum of previous gradients.