Towards Bridging the Gap Between Offline and Iterative Alignment via Preference Distillation
Direct preference optimization DPO is a promising offline approach for aligning large language models (LLMs) due to its simplicity, computational efficiency, and implicit modeling of human preferences. Interestingly, iterative extensions of DPO have achieved stronger performance on academic benchmarks, raising two key...
Wen-Bo Zhang, Wen-Zhuo Zhou, Heng-Rui Cai et al.
· 0 citations