Resist, Update, Reject: Preference Optimization Installs a Prior-Dependent Reliability Switch
It is shown the objective most anti-sycophancy work optimizes is non-identifying with respect to source reliability: because no preference label depends on whether a source is actually reliable, any scalar mixture of the arms traces a single deference dial.
Sen Yang, Yuen-Hei Yeung
· 0 citations