Christian Moya

Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control

In reinforcement learning with verifiable rewards (RLVR), imperfect verifiers can reward incorrect responses, creating opportunities for reward hacking. Using gradient flow with a fixed verifier, we characterize the conditions under which reward rises while correctness falls. We then show that the observations availabl...