Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features
SCALE (Selective Control of Adaptation via Local Entropy), an entropy-guided adaptation-strength-control method that freezes the pretrained model and the SFT delta and learns bounded token- and module-specific gates by minimizing predictive entropy alone is proposed.
Cun-Chun Li, Hao-Nan He, Yi-Fan Gao et al.
· 0 citations