CT-based AI risk support for cystic renal masses: multicenter validation, reader study, and a randomized implementation study
Abstract
Summary Background Cystic renal masses are frequently overtreated. We developed and evaluated the Kidney Imaging-based Triage Engine for Cystic Renal Masses (KITE-CRM), a computed tomography (CT)-based deep-learning pathway using artificial intelligence (AI) to support malignancy risk assessment in cystic renal masses (CRMs). Methods We conducted a multicentre development and validation study in China. In phase 1 (January 2020–December 2021), we developed the system in a retrospective training cohort from a tertiary referral center. CRMs were defined as renal lesions with predominantly cystic features. Corticomedullary-phase contrast-enhanced CT was the sole model input. In phase 2 (January 2022–October 2025), we externally validated the model across five independent hospitals. The core task was lesion-level malignancy discrimination; secondary tasks classified malignant lesions as clear-cell renal cell carcinoma (ccRCC) versus other malignant subtypes and stratified ccRCC as high-grade versus low-grade disease. Retrospective cohorts comprised 3150 patients. The core task used 570 training cases (142 malignant), 235 internal-validation cases (137 malignant), and 528 external cases (160 malignant). Three-dimensional processing used one index lesion per patient and a fixed threshold of 0.586442888. In a randomized crossover multi-reader, multi-case study, six radiologists interpreted 117 external-cohort cases with and without AI; outcomes were area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, reading time, and user acceptance. In phase 3 (November 2025–February 2026), 5869 eligible consecutive adults undergoing contrast-enhanced abdominal CT at Changhai Hospital were randomized 1:1 to AI-assisted (n = 2934) or standard reporting (n = 2935). The primary clinical endpoint was the proportion of pathologically benign lesions among treated lesions. The trial was registered with the Chinese Clinical Trial Registry (ChiCTR2500112290); model code is publicly available. Findings Internal-validation AUROC was 0.996 (95% confidence interval [CI], 0.990–0.999) and Brier score 0.046 (0.036–0.058). External AUROCs were 0.913–0.999, excluding Test 4 (AUROC 1.000; three malignant cases). External sensitivity ranged from 19/24 (79.2%) to 3/3 (100.0%) and specificity from 100/116 (86.2%) to 33/33 and 115/115 (100.0%). AI assistance increased reader-averaged AUROC from 0.825 to 0.973 (difference, 0.149; 95% CI, 0.091–0.206) and reduced mean interpretation time from 9.7 to 5.5 min. Benign pathology occurred in 28/171 (16.4%) treated lesions with AI versus 104/258 (40.3%) with standard care (difference, −23.9 percentage points; 95% CI, −31.7 to −15.4). The treated AI-evaluable CRM subgroup difference was non-significant (5/29 [17.2%] versus 19/56 [33.9%]; p = 0.105). In post hoc intention-to-treat analysis, benign intervention occurred in 28/2934 (0.95%) versus 104/2935 (3.54%) randomized participants (risk difference, −2.59 percentage points; 95% CI, −3.38 to −1.85; p < 0.0001). Median reporting time was 7.6 versus 18.5 min. Interpretation KITE-CRM improved assisted interpretation and reporting efficiency. Management findings do not establish long-term oncologic safety or CRM-specific clinical superiority. The model complements, rather than replaces, Bosniak assessment, indicated multiphase CT or magnetic resonance imaging (MRI), and clinical judgment. Broader prospective evaluation is needed. Funding 10.13039/100014717National Natural Science Foundation of China and Shanghai municipal research programmes.