Zonglin Lyu

Learning from Noisy Preferences: A Semi-Supervised Learning Approach to Direct Preference Optimization

Semi-DPO is proposed, a semi-supervised approach that treats consistent pairs as clean labeled data and conflicting ones as noisy unlabeled data and significantly improves alignment with complex human preferences, without requiring additional human annotation or explicit reward models during training.