Are We Grading Properly? Understanding Failure Modes in Medical Benchmarks
This work applies RIFT, a global rubric failure taxonomy, to two clinical benchmarks (HealthBench Professional and LiveMedBench), and finds failure modes are meaningful: on HealthBench Professional an LLM judge flags 29.6% of criteria as non-atomic and 65.4% as misaligned/rigid.
Prithvi Dixit, Pedram Hosseini
· 0 citations