How Proper Scoring Rules Shape LLM Forecasting
This paper evaluates how reward function choice shapes the performance and behavior of LLM forecasters. We compare five proper scoring rules as training objectives for binary forecasts of resolved real-world events. Although the rules share the same theoretical incentive for truthful probability reporting, the resultin...
Benjamin Turtel, Paul Wilczewski, Kris Skotheim et al.
· 0 citations