SeVeR is proposed, a selective visual exposure framework that compresses dense volumes into modality-wise prototypes and retrieves complementary multi-level evidence with change-aware gated attention during decoding, trained with a marginal-utility self-consistency objective that suppresses unhelpful retrieval.
Abstract
Volumetric medical VQA requires reasoning over long and redundant 3D visual token sequences, especially in multi-sequence MRI where complementary modalities provide diverse diagnostic cues but expose the decoder to many repeated anatomical regions. To investigate reasoning under multi-sequence visual redundancy, we first introduce BreMRIs-VQA, a clinically curated breast MRI benchmark with 1.19M QA pairs from 71.0K sequences and 12.9K patients, covering both free-text and multiple-choice questions. We further propose SeVeR, a selective visual exposure framework that compresses dense volumes into modality-wise prototypes and retrieves complementary multi-level evidence with change-aware gated attention during decoding, trained with a marginal-utility self-consistency objective that suppresses unhelpful retrieval. Experiments on BreMRIs-VQA and public benchmarks show that SeVeR improves both discriminative and generative performance while exposing substantially fewer visual tokens.
A dual-branch frequency-domain fusion module that conditions spectral filtering on the input question, enabling adaptive selection of global low-frequency structure and fine-grained high-frequency detail before reconstructing the spatial representation for answer generation is introduced.
Integrating 3D medical images with vision-language models (VLMs) holds substantial promise for computer-aided diagnosis. However, volumetric images generate prohibitively long visual-token sequences with considerable spatial and inter-slice redundancy. Existing token compression methods typically apply uniform reduction or rely on a single importance signal, increasing the risk of removing regions that are clinically relevant to the query or structurally distinctive. To address this limitation, we propose MedARC, a unified, training-free framework for Adaptive Redundancy Compression of visual tokens in 3D medical VLMs. MedARC estimates token importance by integrating three complementary cues: self-attention from the VLM vision encoder, which reflects the model's intrinsic visual focus; similarity between projected visual tokens and text embeddings, which identifies query-relevant regions; and deviations of local visual foundation model features from the volume-level feature center, which highlight structurally distinctive anatomy. The resulting importance distribution guides a saliency-aware merging strategy that preserves informative tokens while consolidating redundant ones rather than simply discarding them. Experiments on CT-RATE and MR-RATE show that MedARC reduces visual-token overhead and inference time while preserving or improving diagnostic performance. Its multi-cue scoring cost is outweighed by the savings from processing fewer tokens, with greater benefits expected for larger language models.
Yitao Zhu, Meng-Jun Liu, Yingji Fu et al.· arXiv.org· 0 citations
The exponential growth in medical imaging volumes necessitates scalable, reliable diagnostic support systems capable of augmenting clinical workflows. This article presents a systematic quantitative evaluation of state-of-the-art Multimodal Large Language Models (MLLMs) for radiology Visual Question Answering (VQA), a task requiring integrated visual perception and clinical reasoning. We benchmark five leading models — GPT5-Nano, Gemini 3 Flash, Qwen3-VL-8B, LLaVA Next, and Llama 3.2 Vision — on the VQA-RAD dataset under a rigorous zero-shot protocol with standardized prompts and comprehensive precision–recall–F1 evaluation. Our empirical analysis reveals that Gemini 3 Flash achieves superior balanced performance (F1 = 0.78, Accuracy = 0.78, Recall = 0.83), while Qwen3-VL-8B attains the highest precision (0.78) while also maintaining competitive recall. These outcomes demonstrate that general-purpose MLLMs can perform competitively with specialized medical models in tasks such as modality and organ recognition, but still struggle with abnormality detection and complex clinical reasoning. The findings reinforce that MLLMs currently serve best as assistive decisionsupport tools rather than autonomous diagnostic agents, and highlight the potential of retrieval-augmented and context-aware strategies for improving clinical reliability and interpretability.
Cristovão Pessoa Cândido, Matheus Alves de Oliveira Lima, C. de Souza Baptista et al.· International Journal of Sem...· 0 citations
Difference Visual Question Answering (Diff-VQA) in medical imaging automatically compares patient images across time points to support assessment of lesion progression and treatment efficacy. However, pixel-level matching is unreliable due to non-rigid deformations, view shifts, and acquisition noise, while existing models often rely on synthetic labels and lack effective integration of local and global information. To address these challenges, we propose a multimodal large-model framework that adopts a progressive “local semantic modeling–global difference reasoning” strategy. Key anatomical regions in chest X-rays are localized via object detection and aligned with VinDr-CXR annotations to construct region–disease mappings, transforming misalignment into semantic difference analysis. A dynamic sampling strategy further generates clinically meaningful image pairs with fine-grained difference labels. Finally, a multimodal large model fuses local features with global context to support single-image QA, dual-image disease description, and global difference reasoning. Experiments on the MIMIC-Diff-VQA dataset demonstrate state-of-the-art accuracy in single-image QA and substantial improvements in Diff-VQA tasks over mainstream medical large models. In the single-image QA tasks, our model improves accuracy from 52.5% to 64.1% (22.2% relative improvement), and in the Diff-VQA tasks, the CIDEr score increases from 1.027 to 1.379 (34.3% relative improvement). These results highlight the framework’s potential to enhance diagnostic accuracy and strengthen clinical decision support in radiology practice.
Bokai Yang, Haorong Li, Yi-Rong Qin et al.· Information Technology and C...· 0 citations
Access to reliable medical images is essential for clinical training. To address this need, this paper presents an extended version of MIRAGE, a multimodal retrieval and generation system that utilizes a shared latent space to process medical queries by retrieving real images from the ROCO dataset, generating synthetic scans, and providing LLM-based clinical descriptions alongside dual-concept visual comparisons. To overcome previous computational limits and the lack of clinical validation, we introduce three core enhancements: first, an Auto-α module to dynamically weight visual and textual similarities; second, the integration of LCM-LoRA to accelerate synthetic image generation; and third, an automated clinical auditor based on Gemini 2.5 Flash. Experimental results demonstrate that Auto-α improves retrieval accuracy for heterogeneous queries, reaching 38.83% Top-1 Recall over a 65,419-image gallery and outperforming nine fusion baselines evaluated under a unified configuration, with a controlled ablation attributing most of this gain to learning the weight rather than merely making it query-adaptive, while the LCM-LoRA module reduces computational costs by a factor of 12.5× in CPU environments, with a blinded radiologist evaluation confirming only a small drop in clinical quality. Furthermore, the clinical auditor achieves a 0.805 Pearson correlation against an expert radiologist, effectively correcting the systematic overestimation of traditional CLIP scores. Finally, the optimized platform is publicly deployed on Hugging Face.
Miguel Díaz-Benito, Cecilia Diana-Albelda, Álvaro García-Martín et al.· Journal of Imaging· 0 citations
LocAnyMed-CoT-20K is derived, a rationale-augmented subset that connects anatomical context, visual observations, and spatial conclusions through structured reasoning and further improves cross-source generalization through fine-tuning.
Zi-Han Wang, Tong Liu, Zhi-Wei Wang et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
