Frequency-semantic Joint Calibration for Text-guided Medical Image Segmentation

Abstract

Automatically segmenting infected regions in radiological images is crucial for computer-aided diagnosis of pulmonary infectious diseases. Recently, multi-modal methods that integrate clinical text reports as semantic guidance into visual models have demonstrated significant potential in medical image segmentation. However, existing text-guided segmentation approaches still face two primary challenges. First, medical images typically suffer from blurred lesion boundaries and extremely low contrast. Second, existing methods often lack deep decoupling of visual frequency-domain information and precise semantic alignment for visual-linguistic features, leading to an inherent cross-modal semantic gap. To address these issues, a novel multi-modal medical referring image segmentation framework is proposed. Specifically, a frequency-semantic joint calibration module is designed at the encoding stage. This module employs wavelet transform to decouple visual frequency features and interacts with multi-granularity textual semantics, ultimately generating edge-enhanced visual features and precisely localizing calibrated textual features. Furthermore, a prompt-guided attention decoder is constructed to precisely aggregate textual prompts with visual features. Extensive experiments on the QaTa-COV19 and MosMedData+ datasets demonstrate that the proposed framework significantly outperforms existing state-of-the-art methods both quantitatively and qualitatively.