DRIVE: Fine-tuning via Data Contribution- and Diversity-aware Weighting with Prior Regularization
Single-stage fine-tuning is pervasive, yet catastrophic forgetting persists when the source pre-training data is inaccessible. Data weighting affords a natural lever to modulate sample influence. Yet existing weighting methods lack a principled mechanism to explicitly arbitrate knowledge retention against new-task adap...
Qing Liu, Xin-Rui Chen, Yi Du et al.
· 0 citations