A stability aware meta adaptive framework for imbalance handling in machine learning and computer vision

Abstract

Machine learning models are widely used in computer vision and classification tasks. However, imbalanced classification biases predictive models toward larger classes, reducing predictive performance for minority classes. To address this challenge, we propose meta-adaptive resampling selection plus plus (MARS + +), a stability aware meta-adaptive framework that chooses the best method based on the data. It can select from different types, such as resampling, hybrid, and algorithm-based methods. The proposed method uses inner cross-validation to test each option. It selects methods based on both performance and stability by combing predictive performance and variability across folds (mean − λ·std). It also avoids using any method if none gives clear improvement. MARS +  + is tested on multiple benchmark datasets with varying imbalance levels, including both tabular and image data. We use two classifiers, the random forest and logistic regression. The results show that no single method is always the best. MARS +  + still gives results close to the best choice in most cases. It also avoids the large drops in performance seen in some other methods. Statistical tests support the effectiveness of adaptive selection. This shows the importance of selecting methods based on the data. In addition, we provide a detailed analysis of method selection, and also examine cases where no method is selected. Finally, the study analyzes how close the results are to the best possible performance. It also examines how dataset features, such as imbalance and sample size, affect the method selection. These results suggest that selecting methods based on stability worked well on the evaluated datasets. They also support using an adaptive imbalance handling strategy.