LLM judges evaluate candidate language models through pairwise comparisons of their responses. Judge-aware Bradley--Terry models aggregate pairwise comparisons by jointly estimating candidate scores and judge-specific sensitivities (how strongly each judge responds to quality differences), often interpreted as reliabil...
You Liu, Yue Liu, Quan-Chao Lu et al.· 0 citations
The results support reporting permission and recovery together across declared interface variants across declared interface variants, and covers four purposive tasks, fixed GPT endpoints, and one recovery decision.
Judge-specific sensitivity is useful for aggregating pairwise LLM evaluations, but its interpretation depends on which systematic presentation effects the ranking model includes. We introduce OSCAR, an order-aware framework for scoring and calibrating AI rankings, and study position as one such effect. In released judg...
You Liu, Yue Liu, Quanchao Lu et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.