Foundation Models and Vision-Language Models in Medical Imaging: A Review

Abstract

Foundation models (FMs) and vision-language models (VLMs) have emerged as a transformative paradigm in artificial intelligence, reshaping the landscape of medical image analysis and clinical decision support. Building upon large-scale pretraining on heterogeneous datasets, these models exhibit unprecedented generalization and zero-shot transfer capabilities, effectively overcoming the perennial bottleneck of annotated data scarcity that has historically constrained deep learning applications in healthcare. This review paper presents the state of the art in FMs and VLMs applied to medical imaging, encompassing a broad spectrum of modalities including radiography, computed tomography (CT), magnetic resonance imaging (MRI), pathology whole-slide images, ophthalmology, dermatology, and echocardiography. A structured taxonomy of architectures, training strategies, and adaptation mechanisms is provided, covering contrastive learning (CLIP-based), generative, and hybrid paradigms. Also, seminal works are critically analyzed, such as MedCLIP, BioViL, BiomedCLIP, MedSAM, SAM-Med2D, LLaVA-Med, Med-PaLM 2, CONCH, and BiomedGPT, examining their methodological contributions, benchmark performance, and inherent limitations. Additionally, research gaps are discussed, including insufficient 3D volumetric understanding, limited multimodal reasoning in rare disease contexts, translational barriers for clinical deployment, and critical ethical concerns such as algorithmic bias and privacy. Further, emerging trends are presented encompassing instruction-tuned multimodal large language models (LLMs), agent-based systems, federated learning, and FMs for time-series physiological data. Finally, research directions are highlighted for clinically validated, equitable, and deployable AI systems in medical imaging.