A multi-reader multi-case framework for evaluating the decision impact of emerging diagnostic tests for lung cancer
TL;DR
Findings show that the MRMC framework was feasible in a practice-proximate setting and the trends merit further evaluation in larger cohorts and earlier diagnostic settings.
Abstract
We assessed the feasibility of a multi-reader, multi-case (MRMC) observer-performance design to evaluate the simulated decision impact of a radiomics-based computer-assisted diagnostic tool (CADx) and a methylated circulating tumour DNA (ctDNA) assay in lung cancer work-up. Patients referred to a Danish fast-track lung cancer clinic were enrolled. Baseline chest CT scans were analysed with the CADx RevealAI-Lung and plasma with a five-target methylated ctDNA assay. Virtual cases were presented in four information sets: standard of care (SoC), SoC plus ctDNA, SoC plus CADx, and SoC plus both tests. Four pulmonologists independently triaged each case. Diagnostic performance was evaluated using ROC analyses, and triage patterns were compared across information sets. The cohort comprised 239 participants, including 104 lung cancer cases and 135 cancer-free subjects, resulting in 956 unique cases. Both the CADx result and ctDNA positivity were associated with lung cancer. Discrimination was high and similar across information sets (AUC 0.84 to 0.86), with no significant differences. Additional information produced modest, non-significant shifts, mainly among cancer-free subjects, with the combined set tending towards fewer nodule follow-up allocations and more discharges. These findings show that the MRMC framework was feasible in a practice-proximate setting and the trends merit further evaluation in larger cohorts and earlier diagnostic settings.