Detectable, Task-Relevant, or Harmful? Construct-Valid Evaluation of Unlabeled Distribution-Shift Monitoring in Chest Radiography
Distribution-shift alarms do not establish classifier harm. We evaluated the detectability, frozen-head relevance, and performance deterioration of two public DenseNet-121 chest-radiography classifiers under 30 controlled image conditions. Signals covered pixels, 64-component PCA representations, logits, probabilities,...