Do Calibrated-Probability Claims Hold Up in Clinical Diagnosis? A Pre-registered Evaluation of a Decision Model Against Frontier Language Models
Background: A new class of AI "decision models" returns a probability for every option in a single pass and is marketed as calibrated, which invites use in clinical triage. Whether those probabilities hold up on clinical diagnosis has not been independently tested. Objective: To test the diagnostic accuracy, calibratio...
A. Deobhakta, J. Chang, R.-W.-S. Chen et al.
· medRxiv · 0 citations