triprompt: deformation-aware multimodal prompting for robust 3D segmentation

Abstract

Accurate 3D medical image segmentation remains extremely challenging under severe anatomical variability caused by respiration motion, growth, and tumor-induced deformation. Existing prompt-based strategies mainly rely on visual and textual cues that essentially capture what organs look like or what they mean, although they fail to model how organs deform, which is a key determinant of organ geometry. We propose triprompt , a prompt-adaptive segmentation framework that jointly integrates structural visual prompt derived from localized image features; medical text prompt encoding domain knowledge; and a novel Population-Level Deformation Prompt (PDP) that learns statistical organ deformation patterns from data through a compact latent representation. These three prompts are jointly fused via a query-centric triprompt aligner , a lightweight alignment module that performs cross-prompt representation learning, allowing the segmenter to condition simultaneously on appearance, semantics, and physiological deformation. Empirical results on 11 public CT benchmarks show consistent gains in segmentation stability under extreme geometric shifts, underscoring the necessity of deformation-aware prompting for reliable, clinically actionable image segmentation. In order to promote reproducibility, our codes are open-sourced at: https://github.com/llmresearch678/triprompt .