Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning
Domain-specific continual adaptation of LLMs risks catastrophic forgetting, creating a fundamental tension between acquiring new capabilities and preserving those learned during pretraining. PEFT mitigates this problem by restricting the number of trainable parameters, but existing methods lack a principled unit for de...
Ling-Xiang Wang, Hai-Nan Zhang, Liang Pang et al.
· 0 citations