Adaptive Dynamic Feature Selection using Evolutionary Meta-Learning for Streaming Data

Abstract

This paper presents an Evolutionary Meta-Learning Framework (EMLF) for dynamic feature selection in high-velocity, non-stationary data streams. Unlike static methods, EMLF continuously updates the selected feature subset as feature relevance changes over time. A multi-objective fitness function, normalized using prediction accuracy, inference latency, and memory usage, evaluates candidate subsets represented as binary chromosomes. The framework also fine-tunes the probabilities of evolutionary operators using an exponentiated-gradient meta-update based on observed fitness gains. Concept drift is explicitly detected using ADWIN, triggering the reevaluation of feature relevance. EMLF was evaluated on the Intel Berkeley Research Lab Sensor stream using an interleaved prequential protocol across 30 independent runs. It achieved an accuracy of 92.60 ± 0.32% and an F1-score of 91.40 ± 0.37%, outperforming the best baseline by 2.60 percentage points. It also achieved an inference latency of 135 ms per mini-batch, representing a 22.9% reduction compared with the best baseline, while reducing the number of input features by 37.5%. On an independent 128-feature benchmark, EMLF achieved 96.27% accuracy compared with 93.42% for the second-best-performing benchmark. Overall, EMLF demonstrates a strong trade-off between predictive accuracy, computational complexity, and adaptability; however, its evaluation is limited to two datasets and a single hardware configuration.