How Learning Governs Unlearning across the Memorization-Generalization Spectrum
This paper first classify memorization- and generalization-heavy models using grokking in modular addition and compare their responses to unlearning, showing that the latter suffer greater retain damage, i.e., a larger performance drop on the retain set.
Hwiyeong Lee, H. Lim, Ingyu Bang et al.
· 0 citations