A Segmentation-Assisted Dual-Branch BioMedCLIP Approach for Thyroid Ultrasound Classification
TL;DR
A segmentation-assisted dual-branch BioMedCLIP architecture is proposed on the recently published, pathology-validated ThyroidXL dataset, which includes over 4,000 patients, and achieves significantly outperforming standard deep learning models in the literature.
Abstract
The ultrasound-based classification of thyroid nodules into benign and malignant categories is an important problem in clinical decision support. However, it remains challenging because benign and malignant nodules may show similar visual characteristics, and ultrasound interpretation is often affected by operator-dependent variability. To address these challenges, a segmentation-assisted dual-branch BioMedCLIP architecture is proposed on the recently published, pathology-validated ThyroidXL dataset, which includes over 4,000 patients. The original image provides global anatomical context, while the segmentation-masked image emphasizes the nodule area and reduces the influence of surrounding tissues. These two inputs are processed by a shared-weight BioMedCLIP visual encoder pre-trained on medical data, and the resulting feature representations are combined before the final classification layer. Experimental results demonstrate that the proposed architecture achieves a 0.9699 F1-score and 0.9939 AUC at the patient level, significantly outperforming standard deep learning models in the literature (e.g., VGG-13, EfficientNet).