Prior-Gated Adaptation With Multi-Granularity Structural Refinement for Fine-Grained Infrared Small-Target Segmentation

Abstract

Accurate infrared small-target segmentation is an inherently challenging low signal-to-noise ratio task, where targets occupy only a few pixels and are often obscured by weak responses, ambiguous boundaries, and complex clutter. Although vision foundation models offer transferable representations, their direct adaptation to fine-grained infrared segmentation remains difficult: full fine-tuning is costly, while generic parameter-efficient adapters lack infrared-specific constraints and may enhance clutter together with weak targets. To address this issue, we propose PGSRNet, a prior-gated adaptation framework with multi-granularity structural refinement. Specifically, a frozen InfMAE encoder is adapted through the proposed prior-gated residual adapter (PGRA), which leverages local contrast, edge, and background-complexity priors to control where and how strongly lightweight residual updates are applied, thereby enhancing weak targets while suppressing clutter responses. To further preserve small-target structures, PGSRNet incorporates an image-level boundary-aware multisource refinement (BMSR) module for contour correction and a patch-level local detail refinement (PLDR) module for fine-detail recovery within candidate target regions. Experiments on two real-world infrared small-target datasets demonstrate that PGSRNet achieves intersection over union (IoU)/ $F1$ scores of 71.38%/83.30% on IRSTD-1k and 75.78%/86.22% on NUAA-SIRST, outperforming the InfMAE-pretrained UPerNet baseline by 5.68%/4.00% and 10.03%/6.88%, respectively. With 22.51M trainable parameters, PGSRNet also shows competitive performance against recent state-of-the-art methods, demonstrating the effectiveness and efficiency of infrared prior-gated adaptation for fine-grained small-target segmentation. Code is available at https://github.com/zhilyzhang/PGSRNet.