#machine learning
Jul 2026
On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
This paper proposes the concept of non-hackable confidence reward schemes and defines a spectrum of such reward schemes for RL confidence calibration training in LLMs, and demonstrates that selective confidence reward hacking can occur in practical datasets with reward schemes that are not designed to be non-hackable.
C. Tan, Zhuoyi Lin, M. Motani et al.
· arXiv.org · 0 citations