A Hybrid Genetic Algorithm with Information-Theoretic Local Search for Unsupervised Feature Selection

Abstract

: Feature selection (FS) plays a crucial role in machine learning by reducing data dimensionality and improving learning efficiency. In many real-world scenarios, label information is unavailable, making unsupervised FS particularly important. While Genetic Algorithm (GA) offers a powerful global search mechanism for subset selection, it often suffers from premature convergence and struggles to refine solutions in complex search spaces. To address these limitations, we propose a hybrid GA that integrates an information-theoretic local search strategy for unsupervised FS. The proposed method integrates an information-theoretic local refinement procedure, consisting of DEL and ADD operations based on joint entropy, into a conventional GA framework. Unlike conventional evolutionary methods, our approach leverages information-theoretic measures not merely for evaluation, but as a guiding mechanism for fine-grained local exploration within the GA framework. By incorporating mutual information-based local refinement, the proposed method effectively overcomes the convergence bottlenecks of standard GAs, ensuring a more robust exploitation of feature dependencies. Experimental results on five datasets demonstrate that the proposed method consistently achieves higher clustering performance compared with conventional methods. These results imply that the proposed information-theoretic local refinement effectively mitigates the premature convergence problem of conventional GAs and improves search efficiency and solution quality compared to traditional heuristic and evolutionary approaches. It provides a promising framework for handling high-dimensional data in scenarios where label information is unavailable.