Adaptive Diffusion Vision-Language Models for Reliable Medical Image Understanding

Abstract

Biomedical vision–language models increasingly support image-grounded clinical dialogue, yet most deployable systems still depend on autoregressive language generation. Such systems tend to truncate answers, react poorly to length instructions, and offer no principled way to signal uncertainty when image evidence is weak. We present MedDiffVL, a biomedical vision-language model that pairs a masked language diffusion backbone with a SigLIP-2 visual encoder and a multimodal alignment pipeline that injects modality and question-type cues. Three inference-time mechanisms target the failure modes of diffusion-based generators in the clinical setting. An adaptive confidence-guided remasking rule uses a time-aware threshold and a short-window stability check to remove repetitive low-quality candidates. A clinically aware length controller selects a target length from question type, modality, and an internal uncertainty estimate. A reliability gate combines visual-evidence and answer-confidence scores to emit, hedge, or escalate a response. On VQA-RAD, SLAKE, and PathVQA, the model reaches 85.42, 92.78, and 94.91% closed-form accuracy and an overall conversation score of 53.42 against a fixed reference. Token repetition falls from 0.18 to 0.06. An ECE falls from 0.137 to 0.034, but this reflects an ECE-surrogate training loss and is not independently validated. These gains are not uniform. The closed-form gains over the prior diffusion model lie within run-to-run variance, and latency stays higher than autoregressive baselines. The main contribution is controllability and reliability-aware decoding, not higher closed-form accuracy. The results indicate that confidence-guided masked diffusion with reliability-aware decoding is a useful direction for controllable and reliability-aware clinical assistants.