Jul 2026
Metadata-Free Meta-Reweighted Direct Preference Optimization under Noisy Preference Labels
A bilevel optimization framework is proposed and it is proved, under some idealized conditions, that this framework can recover the DPO optimum under clean data and derive a prior form for the learnable weighting function under label-flipping noise.
Hua Qu, Yifan Li, Xiaodong Yuan
· arXiv.org · 0 citations