ViCoR: Estimating Visual Necessity via Counterfactual Residuals for Multimodal Medical Data Selection
Medical vision-language models often exhibit hidden stratification, where high overall performance masks failures on subtle visual findings and overlapping clinical conditions. We trace this failure mode to modality imbalance during fine-tuning, where strong linguistic priors can suppress visually grounded learning and...