The Format is the Filter: Scoring Format Fixes What an LLM Judge Can Express
When an LLM judge filters instruction-tuning data, its scoring format (binary verdict, 1--5 rating, 0--100 score, or comparison) is rarely justified or ablated. Holding the candidate pool, judge prompt, decoding and selection budget fixed, we score 15,406 UltraFeedback candidates under five formats with three judges, c...
Lovedeep Gondara, Jekaterina Novikova, S. Suresh et al.
· 0 citations