A Unified Self-Supervised CNN–Transformer Architecture for Joint Image Enhancement and Segmentation Under Label Scarcity
Abstract
Visual analysis is essential in practice, and image enhancement and segmentation are fundamental elements of it, but most deep learning models address these problems separately and use large annotated datasets. The paper introduces a single self-supervised hybrid CNN Transformer model that performs joint image enhancement and segmentation in the case of label scarcity. The suggested model uses convolutional layers to extract local features and Transformer modules to model global contextual features. It applies a self-supervised pretraining approach to unlabeled data based on the use of enhancement-guided and structural consistency goals, which results in the effective learning of representations without manual labels. Fine-tuning of the pretrained model to segmentation is performed using a few labeled samples under the same unified model. Experimental findings on state-of-the-art datasets prove that the suggested method positively affects the segmentation performance measured by Dice score by 3 to 6 percent relative to fully supervised baselines in low-label conditions, which also can be characterized by significant improvements in image quality metrics, including PSNR and SSIM. These findings prove superiority of joint self-supervised learning in strong and generalizable image enhancement and segmentation.