An Evidence-based Analysis of Mask Selection in Zero-Shot Medical Image Segmentation

Abstract

Medical image segmentation plays an important role in clinical imaging for disease diagnosis, treatment planning, and follow-up analysis. Zero-shot segmentation models, like segment anything model are able to perform well but ultrasound images remain challenging due to speckle noise, low contrast and poor anatomical boundaries. Furthermore, when these models generate multiple masks per image, identifying which mask provides the best segmentation results can be difficult. The candidate mask selection problem in zero-shot medical image segmentation was considered limited detail by previous research. Therefore, this paper presents a lightweight arbitration framework that identifies clinically relevant masks from multiple candidates using a standard segmentation model, without retraining or modifying the model. The approach evaluates candidate masks using soft anatomical and geometric constraints such as region size, structural compactness, and boundary interaction. Experiments were conducted on the BUSI breast ultrasound dataset under strict zero-shot conditions without supervised fine-tuning. Results indicate that rigid anatomical assumptions reduce segmentation reliability in highly variable ultrasound images, while softer geometric constraints improve robustness during mask selection. The findings further show that boundary-aware anatomical reasoning provides more consistent candidate selection than relying only on global spatial priors.