Decomposing LLM-Judge Uncertainty to Target Expert Labels
This work demonstrates it can estimate where a judge is ignorant rather than where experts genuinely disagree, and proposes using this to direct expert labelling.
3 papers indexed here
We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.
Not the right person? Other researchers publish under this name.
This work demonstrates it can estimate where a judge is ignorant rather than where experts genuinely disagree, and proposes using this to direct expert labelling.
Audited three commercial AI scribes on the same 142 consultations: 565 notes from recorded UK primary-care and US ambulatory encounters plus authored scenarios, with one failure mode drawn from published scribe-error taxonomies.
It is asked whether judges detect omissions in clinical notes, and two methods reach it independently and trade off: a per-fact pipeline, and a GEPA-evolved prompt doing the same in one call.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.