Beyond Aggregate Scores: Behavioral Correctness Assumptions for Assessing Reference-Based Automatic Evaluation Methods
It is demonstrated that behavioral correctness assumptions provide diagnostic information obscured by conventional aggregate meta-evaluation, and no evaluator satisfies all proposed correctness assumptions, and evaluators with similar aggregate performance can exhibit substantially different behavioral profiles.
Maria Mahbub, Ashley Rice, Michael R. Munroe et al.
· 0 citations