Supervised Feature Selection via Collective First-Order Neural Dynamics.

Abstract

Feature selection is a core machine learning technique for selecting the most informative features. Most existing feature selection methods rely on greedy strategies, leading to solutions. Although some global feature selection methods improve classification performance, their effectiveness remains limited due to category information or inaccurate relevance measurements. In this article, the feature selection task is reformulated as a mixed-integer optimization problem that minimizes feature redundancy and maximizes feature relevance. Given the nonconvex nature of this formulation, a collective first-order neural dynamics (CFND) method is proposed to solve the optimization problem. The method combines first-order neural dynamics (FND) with a collective mechanism to substantially enhance the global search capability. Theoretical analyses prove that the proposed CFND method converges globally with probability 1. Extensive experiments across ten benchmark datasets demonstrate that the proposed CFND method consistently outperforms other feature selection methods across all classifiers, achieving an average accuracy improvement of approximately 2%-5%. Compared with traditional methods, the maximum performance gain exceeds 6%. These findings validate that the synergy between FND and the collective mechanism effectively balances local exploitation and global exploration, leading to robust feature selection performance across diverse datasets and parameter settings. The source code is available at https://github.com/LongJin-lab/CFND.