Skip to content

MetaStructAtlas: A Grounded 3D Vision-Language Dataset and Benchmark for Functional and Structural Reasoning in Whole-Body PET/CT

Sep 2026 · arXiv.org · Vol abs/2609.03690 · 0 citations · 33 references
Computer Science

TL;DR

This work introduces MetaStructAtlas, a large-scale dataset for grounded whole-body PET/CT interpretation that synthesizes multimodal imaging with integrated anatomical, metabolic, and semantic annotations and further developed MetaStructVQA, a standardized 3D grounded visual question-answering benchmark containing 100,565 QA pairs.

Abstract

The joint interpretation of metabolic function and anatomical structure is essential for clinical diagnosis in whole-body PET/CT. Although recent advances in 3D medical vision-language models have demonstrated remarkable progress, current efforts are limited to regional CT imaging, leaving a critical void in comprehensive whole-body PET/CT analysis. In this work, we introduce MetaStructAtlas, a large-scale dataset for grounded whole-body PET/CT interpretation that synthesizes multimodal imaging with integrated anatomical, metabolic, and semantic annotations. MetaStructAtlas provides 490 co-registered 3D PET and CT volumes with 50,470 organ-level segmentation masks and grounded radiology reports. To facilitate interactive reasoning, we further developed MetaStructVQA, a standardized 3D grounded visual question-answering benchmark containing 100,565 QA pairs. This framework explicitly links diagnostic queries to visual evidence across modalities, encompassing anatomical, morphological, and metabolic characteristics. Finally, we evaluate state-of-the-art 3D medical VLMs on MetaStructVQA, establishing a robust foundation for multimodal representation learning and integrated whole-body reasoning in nuclear medicine.

View source

Similar papers

Open access Oct 2026

UniPET: a unified approach for whole-body CT to PET translation

UniPET is introduced, a unified approach for whole-body CT-to-PET translation based on curriculum learning that provides a promising approach for augmenting CT-based workflows with synthetic metabolic information, with potential applications in screening triage, in low-resource environments where PET is unavailable, an...

Francesco Di Feola, V. Guarrasi, Mikael Johansson et al. · 0 citations
Review Open access Oct 2026

From task-specific AI to multimodal foundation models in PET/CT: a readiness framework for clinical translation

Artificial intelligence in PET/CT is evolving from narrowly optimized systems toward reusable pretrained representations and multimodal models. PET/CT is a demanding test case because clinically meaningful interpretation must preserve quantitative molecular information while integrating anatomic CT, acquisition metadat...

A. Muneer, Maliazurina B. Saad, Hui Xu et al. · 0 citations
Review Open access Jul 2025

CADS: A Comprehensive Anatomical Dataset and Segmentation for Whole-Body Anatomy in Computed Tomography

Volumetric computed tomography (CT) segmentation supports quantitative image analysis, but most AI-based methods remain organ-specific, with fragmented models, inconsistent label definitions, and heterogeneous training and evaluation protocols. Here we present CADS, a unified data–model framework for scalable and gen...

Murong Xu, Fernando Navarro, Maksym Fritsak et al. · 11 citations · ⚡1
#computer vision Review Sep 2026

NV-Reason-CT: 3D Visual Language Model for CT Analysis

The NV-Reason-CT model, a generative vision--language model for chest and abdominal CT combining native 3D visual encoding with radiologist-guided reasoning, and the model and training code are released to support reproducible research on explainable AI for volumetric medical imaging.

Andriy Myronenko, Dong Yang, Yu-Cheng Tang et al. · 0 citations
Review Open access Sep 2026

From 2D Vision–Language Models to Volumetric Medical AI: Large Language Models and Foundation Models for 3D Medical Imaging

Multimodal large language models (MLLMs) and vision–language models (VLMs) have rapidly entered medicine, demonstrating promising performance in clinical reasoning, radiology report generation, and visual question answering (VQA). However, many current multimodal architectures and pretrained visual backbones remain fun...

Roni Ramon-Gonen, Haya Engelstein · 0 citations
Open access Nov 2026

FASA-Net: A Frequency-Domain-Enhanced and Anatomically Constrained Vision-Language Framework for Automated Chest X-ray Report Generation.

Automated radiology report generation can reduce radiologists' workloads by producing preliminary reports from medical images. However, generating chest X-ray reports remains challenging. Subtle lesions and organ boundaries may be underrepresented, visual regions may be poorly aligned with medical terminology, and long...

Yu-Hai Yu, Jing Liu, Jia-Na Meng et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.