SPOTnet: weakly-supervised specular reflection detection and removal for minimally invasive surgery

Abstract

Specular reflections are a common artefact in minimally invasive surgery, obscuring anatomical landmarks and degrading downstream computer vision systems relied upon for artificial intelligence-assisted surgical analysis. Existing approaches either require labour-intensive annotation or fail to generalise across heterogeneous surgical imaging. Here, we present SPOTnet (Specular Pseudo-label Optimised Training Network), a weakly-supervised framework for specular reflection detection and removal that requires no manual labelling. Our three-stage pipeline automatically builds a library of specular exemplars using hue–saturation–value (HSV) colour-space detection with morphological refinement, trains a lightweight U-Net on the resulting 500,000 patch-mask pairs, and integrates Large Mask Inpainting (LaMa) for restoration. Developed on TEMSET-24K (24,306 transanal endoscopic microsurgery frames), SPOTnet attains an IoU of 0.613, a recall of 0.740, and an E-measure of 0.963 on the annotated test set, outperforming recent methods including EndoSRR, Li et al., and UnReflectAnything, and demonstrates cross-dataset generalisation across four independent benchmarks without retraining. Specular removal improved downstream semantic segmentation (mean F1 +26.9%, p<0.01) on TEMSET-24K and polyp segmentation F1 from 0.938 to 0.956 on EndoScene. Prospective evaluation involving 13 surgeons (207 assessments) supported clinical acceptability, with 88.9% rating the processed video as trustworthy. SPOTnet is a scalable, annotation-free pre-processing step that improves downstream surgical computer vision.