Jaeho Han

The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models

A calibrated Vision-Language Model (VLM) can repeatedly self-correct, say"Wait, I should recheck,"arrive at the wrong answer, and still report high confidence. We find that this occurs because verbalized confidence is largely trajectory-independent in the VLMs and calibration methods we evaluate. We examine this throug...