Hybrid Ensemble and Imbalance-Aware Machine Learning for Multi-Class Cardiovascular Disease Severity Prediction

Abstract

Cardiovascular disease prediction using structured clinical data is commonly formulated as a binary classification problem, while multi-class prediction is more challenging because of overlapping clinical characteristics and class imbalance. This study presents a hybrid ensemble and imbalance-aware machine-learning framework for binary and five-class heart-disease prediction using the combined UCI Heart Disease dataset comprising 920 records from Cleveland, Hungary, Switzerland, and VA Long Beach. Six classifiers—k-Nearest Neighbors, Decision Tree, Logistic Regression, Support Vector Machine, Random Forest, and XGBoost—are evaluated together with majority-voting and stacking ensembles. To address the highly imbalanced five-class distribution, SMOTE is applied exclusively to the training folds during stratified cross-validation. A confidence-aware hierarchical voting mechanism is also introduced to resolve classifier disagreements by retaining majority decisions when consensus exists and using posterior-confidence information in ambiguous cases. On the original imbalanced five-class data, XGBoost achieves the highest accuracy of 86.74%. After SMOTE-based balancing, stacking achieves the best overall performance, with 94.94% accuracy, 94.92% precision, 94.94% recall, and 94.92% F1-score, closely followed by majority voting. Feature-importance analysis identifies chest-pain type, maximum heart rate, ST depression, and number of major vessels as influential predictive attributes. The results demonstrate the potential of imbalance-aware heterogeneous ensembles for robust multi-class heart-disease prediction, while the proposed confidence-aware voting strategy provides an explicit mechanism for resolving classifier disagreement.