Locating Sycophancy Across Training Stages in Open Language Models
Language model assistants sometimes change a correct answer when the user suggests a wrong one. The behavior, called sycophancy, is usually attributed to preference-based post-training, yet early measurements found it in pretrained models. We ask where it comes from by measuring it at every available stage of training:...
Jiarui Yao, Shweta Parihar, Nghia Bui et al.
· 0 citations