George Wang

Patterning in Practice: Debiasing Reward Models with Susceptibilities

This paper uses patterning, which reweights each preference pair according to its measured effect on posterior expectation values of benchmark losses (its susceptibility), to debias a Gemma 2 9B Instruct reward model trained on Skywork-Reward-Preference v0.2.