Uncertainty-Aware Prediction Across Endoscopic Domains: Laryngeal Narrow-Band and Gastrointestinal Imaging
Background: Deep-learning systems for endoscopic image classification are commonly evaluated with random data splits, which may overestimate performance under acquisition shift; uncertainty-aware selective prediction may improve reliability by allowing a model to abstain on uncertain cases. Methods: We evaluated binary abnormality detection in two endoscopic imaging domains: laryngeal contact-endoscopy narrow-band imaging (CE-NBI; 210 patients, patient-level) and gastrointestinal endoscopy (HyperKvasir; 6746 images, image-level, predominantly white-light). ImageNet-pretrained ResNet-50 deep ensembles were assessed under a resolution-defined acquisition-shift stress test; for the gastrointestinal data, a random stratified split was additionally used as an in-distribution reference. We evaluated discrimination, calibration, decision-curve analysis, and entropy-based selective prediction. Results: In the gastrointestinal dataset, random-split evaluation produced high performance (AUROC 0.994, 95% CI 0.990–0.996; AUPRC 0.991). Under resolution shift on the same data, performance fell to AUROC 0.723 (95% CI 0.710–0.736; AUPRC 0.690; sensitivity 0.417; specificity 0.874); the two intervals do not overlap. Selective prediction improved reliability among retained cases: under resolution shift, accuracy rose from 0.683 at full coverage to 0.852 (95% CI 0.835–0.871) at 25% coverage (balanced accuracy 0.646 → 0.798). Predictive entropy was significantly higher for incorrect than for correct predictions in both regimes (Mann–Whitney p = 7.1 × 10−77 with rank-biserial |r| = 0.30 under shift). In the laryngeal cohort, no statistically significant differences were detected among four architectures (ROC-AUC 0.844–0.901; all pairwise DeLong p > 0.05). Conclusions: Random-split evaluation substantially overestimated performance relative to a resolution-defined acquisition-shift stress test, and entropy-based selective prediction improved reliability by identifying a high-confidence subset for automated prediction while deferring the remainder to human review. Target-domain recalibration substantially restores calibration under shift (ECE 0.172 → 0.036 with temperature scaling; → 0.017 with isotonic regression) but does not recover discrimination; selective prediction is complementary, mitigating residual confident-wrong predictions. An encoder-transfer experiment showed asymmetric cross-domain utility; features learned on the larger gastrointestinal cohort transferred to the laryngeal cohort (AUROC 0.80 vs. in-domain 0.89), whereas the reverse direction did not transfer (0.53 vs. 0.72). Prospective multi-center validation remains required before clinical deployment. All code, fold definitions, random seeds, and a reproducible protocol are publicly released.