Calibrating LLM Judges for Human and AI Conversations
This work evaluates state-of-the-art LLMs as pointwise and pairwise judges of conversational success on CANDOR, finding pointwise scoring correlates moderately with human ratings, while pairwise comparison suffers from long transcripts and positional bias.
Maike Zufle, Patrícia Schmidtová, Vilém Zouhar et al.
· 0 citations