From Generic to Dedicated: A Novel Optimizer for Online Continual Learning
This work reveals a new link between optimization dynamics and OCL needs by recasting Polyak-Ruppert averaging as the engine of"plasticity" and Primal averaging as the anchor of"stability" in SPIN, which explicitly decouples plasticity and stability by interpolating between a fast-moving"plasticity"sequence and an adap...