Beat-Level Echocardiographic Clip Analysis with Geometry-Video Fusion and Cross–Attention for Accurate Ejection Fraction Estimation and Heart Failure Stratification
TL;DR
The proposed framework, combining beat-level motion analysis and geometric features with Shapley Additive Explanations- and Gradient-Weighted Class Activation Mapping-based explainability, enables quantification of ejection fraction and clinically reliable heart failure classification, thereby scaling echocardiographic decision-making.
Abstract
Left ventricular ejection fraction (LVEF) is a crucial indicator of cardiac function in the diagnosis and management of heart failure. Recent deep learning approaches have shown promising results for automated LVEF estimation from echocardiographic videos; most studies rely on spatiotemporal information, and few studies investigate the contribution of anatomical descriptors. In this study, we propose a comprehensive beat-level framework for EF estimation and heart failure classification using echocardiographic videos. Representative cardiac beat clips were extracted using tracing-derived annotations. Three architectures were investigated: (a) EchoEF-Net, a spatiotemporal Convolution Neural Network (CNN), which is a video-only model; (b) GeoFusionEF-Net, a multimodal deep learning feature-level fusion of geometry and video features for enhanced robustness and calibration; and (c) CrossAttnFusionEF-Net, which employs a staged cross-attention multimodal architecture to integrate the video and geometry interactions adaptively. Experiments were conducted on the publicly available EchoNet-Dynamic dataset (10,030 videos). On the official test set, the proposed GeoFusionEF-Net model effectively captures interactions between the video and geometry modalities, significantly reducing mean absolute error from 4.61 (EchoEF-Net) and 1.99 (CrossAttnFusionEF-Net) to 0.648, while improving root mean squared error and Pearson correlation to 1.413 and 0.99, respectively. The estimated EF predictions were further utilized to automate heart failure stratification into reduced (HFrEF), mid-range (HFmrEF), and preserved (HFpEF) EF categories. The meta-classifier of all three EF estimators achieved 0.97 accuracy and 0.99 Area Under the Curve (AUC). The proposed framework, combining beat-level motion analysis and geometric features with Shapley Additive Explanations- and Gradient-Weighted Class Activation Mapping-based explainability, enables quantification of ejection fraction and clinically reliable heart failure classification, thereby scaling echocardiographic decision-making. Received: 2 April 2026 | Revised: 15 June 2026 | Accepted: 24 June 2026 Conflicts of Interest The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in the EchoNet-Dynamic repository at https://stanfordaimi.azurewebsites.net/datasets/834e1cd1-92f7-4268-9daa-d359198b310a. Author Contribution Statement Chaithra C. S.: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Siddesha S.: Conceptualization, Validation, Formal analysis, Investigation, Writing – review & editing, Supervision, Project administration. Vinay Kumar N.: Validation, Formal analysis, Writing – review & editing. V. N. Manjunath Aradhya: Validation, Formal analysis, Writing – review & editing.