An Unsupervised Transformer-based Clustering Network for Medical Image Segmentation

Abstract

Accurate and reliable medical image segmentation is fundamental to modern clinical diagnosis, treatment planning, and disease monitoring. It remains challenging due to organ shape variability, low contrast, long-range dependencies, and limited annotations, particularly across heterogeneous modalities, MSD spleen CT scan and PROMISE12 prostate MRI. To address these challenges, we propose a novel unsupervised transformer-based clustering network, UTC-Net, that minimizes reliance on labeled data without compromising the performance of the network. UTC-Net incorporates transformer-based feature mapping with spectral graph modeling and K-means clustering, supported by pseudo-label refinement, region-affinity modeling, and augmentation-based regularization. Experimental results demonstrate that UTC-Net achieves superior performance on both datasets while requiring significantly fewer parameters than state-of-the-art networks. An ablation study further exhibits the impact of each important module in the proposed UTC-Net network. Our proposed unsupervised segmentation scheme also demonstrates strong adaptability across modalities and annotation, offering a promising direction for a scalable, lightweight, and annotation-free medical image segmentation scheme.