Direct Preference Density Alignment for Conversational Audio Equalization
Large Language Model alignment typically relies on learned proxy reward models, which significantly increase the memory footprint during training and are notoriously prone to instability and reward hacking. While offline methods like Direct Preference Optimization (DPO) bypass the reward model, they lose the ability to...
Ioannis Stylianou, S. Shepstone, Jon Francombe et al.
· 0 citations