Multimodal Deep Learning Integrating Electrocardiography and Chest Radiography Enhances Prediction of Incident Regurgitant Valvular Heart Disease
TL;DR
A multimodal deep learning model integrating ECG and CXR significantly improved prediction of incident rVHD compared with ECG alone, with the greatest benefit observed for AR.
Abstract
Background: Regurgitant valvular heart disease (rVHD) is a major cause of cardiovascular morbidity. Echocardiography is the diagnostic standard but is resource-intensive for large-scale screening. Electrocardiography (ECG) has shown promise for predicting incident rVHD, yet performance varies across phenotypes, particularly for aortic regurgitation (AR). Chest radiography (CXR) provides complementary structural and hemodynamic information. We hypothesized that a multimodal model integrating ECG and CXR would improve prediction of incident moderate-to-severe rVHD. Methods: In this retrospective multicenter study, we identified 212,888 paired ECG-CXR examinations from 116,380 patients across two Chinese centers. Baseline ECG and CXR were obtained within 60 days of echocardiography. Outcome was progression to moderate-to-severe AR, mitral regurgitation (MR), or tricuspid regurgitation (TR). We developed a multimodal neural network with pretrained unimodal encoders, token-level cross-modal fusion, and a class-specific gating mechanism that adaptively weighted ECG-only, CXR-only, and fused predictions. Performance was assessed using C-index, AUROC, AUPRC, decision curve analysis, net reclassification improvement (NRI), and Kaplan-Meier stratification. Results: Multimodal fusion consistently outperformed unimodal models across all phenotypes. For AR, C-index improved from 0.616 (ECG-only) to 0.713 (multimodal; AUROC 0.729, AUPRC 0.972). For MR, multimodal C-index was 0.801 (AUROC 0.814, AUPRC 0.972), versus 0.782 for ECG and 0.775 for CXR alone. For TR, multimodal and CXR-only models showed similar discrimination (C-index 0.802), but multimodal fusion yielded greater net benefit on decision curve analysis. NRI was positive across all time horizons (1-5 years) for all valve types. Grad-CAM interpretability analyses revealed that ECG attention localized to leads II, V-V (AR), leads I, II, aVF, V-V (MR), and inferior/right precordial leads (TR); CXR attention highlighted chamber-specific enlargement and pulmonary congestion patterns consistent with pathophysiology. Conclusion: A multimodal deep learning model integrating ECG and CXR significantly improved prediction of incident rVHD compared with ECG alone, with the greatest benefit observed for AR. The model leveraged complementary electrical and structural information, demonstrated biological plausibility through interpretability analyses, and provided consistent clinical utility. Given the widespread availability and low cost of both modalities, this approach offers a scalable tool for risk stratification in routine care. Prospective studies are warranted to validate clinical implementation.