This work introduces MetaStructAtlas, a large-scale dataset for grounded whole-body PET/CT interpretation that synthesizes multimodal imaging with integrated anatomical, metabolic, and semantic annotations and further developed MetaStructVQA, a standardized 3D grounded visual question-answering benchmark containing 100,565 QA pairs.
Abstract
The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advances in 3D medical vision-language models have demonstrated remarkable progress, current efforts are limited to regional CT imaging, leaving a critical void in comprehensive whole-body PET/CT analysis. In this work, we introduce MetaStructAtlas, a large-scale dataset for grounded whole-body PET/CT interpretation that synthesizes multimodal imaging with integrated anatomical, metabolic, and semantic annotations. MetaStructAtlas provides 490 co-registered 3D PET and CT volumes with 50,470 organ-level segmentation masks and grounded radiology reports. To facilitate interactive reasoning, we further developed MetaStructVQA, a standardized 3D grounded visual question-answering benchmark containing 100,565 QA pairs. This framework explicitly links diagnostic queries to visual evidence across modalities, encompassing anatomical, morphological, and metabolic characteristics. Finally, we evaluate state-of-the-art 3D medical VLMs on MetaStructVQA, establishing a robust foundation for multimodal representation learning and integrated whole-body reasoning in nuclear medicine.
UniPET is introduced, a unified approach for whole-body CT-to-PET translation based on curriculum learning that provides a promising approach for augmenting CT-based workflows with synthetic metabolic information, with potential applications in screening triage, in low-resource environments where PET is unavailable, an...
Francesco Di Feola, V. Guarrasi, Mikael Johansson et al.· Frontiers in Artificial Inte...· 0 citations
Artificial intelligence in PET/CT is evolving from narrowly optimized systems toward reusable pretrained representations and multimodal models. PET/CT is a demanding test case because clinically meaningful interpretation must preserve quantitative molecular information while integrating anatomic CT, acquisition metadat...
A. Muneer, Maliazurina B. Saad, Hui Xu et al.· Frontiers in Nuclear Medicin...· 0 citations
Volumetric computed tomography (CT) segmentation supports quantitative image analysis, but most AI-based methods remain organ-specific, with fragmented models, inconsistent label definitions, and heterogeneous training and evaluation protocols. Here we present CADS, a unified data–model framework for scalable and gen...
Murong Xu, Fernando Navarro, Maksym Fritsak et al.· Nature Communications· 11 citations· ⚡1
The NV-Reason-CT model, a generative vision--language model for chest and abdominal CT combining native 3D visual encoding with radiologist-guided reasoning, and the model and training code are released to support reproducible research on explainable AI for volumetric medical imaging.
Multimodal large language models (MLLMs) and vision–language models (VLMs) have rapidly entered medicine, demonstrating promising performance in clinical reasoning, radiology report generation, and visual question answering (VQA). However, many current multimodal architectures and pretrained visual backbones remain fun...
Roni Ramon-Gonen, Haya Engelstein· Computation· 0 citations
Automated radiology report generation can reduce radiologists' workloads by producing preliminary reports from medical images. However, generating chest X-ray reports remains challenging. Subtle lesions and organ boundaries may be underrepresented, visual regions may be poorly aligned with medical terminology, and long...
Yu-Hai Yu, Jing Liu, Jia-Na Meng et al.· Medical Engineering and Phys...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.