Mediocrity is the key for LLM as a Judge Anchor Selection
This work systematically investigate the effect of anchor selection by evaluating 22 different anchors on the Arena-Hard-v2.0 dataset, finding that the choice of anchor is critical: a poor anchor can dramatically reduce correlation with human rankings.
Shachar Don-Yehiya, Asaf Yehudai, Leshem Choshen et al.
· Annual Meeting of the Associ... · 6 citations
· ⚡1