A Surrogate-Assisted Synergistic Co-evolutionary Framework for Computationally Expensive Neural Architecture Search

Abstract

Neural Architecture Search (NAS), often formulated as a bilevel optimization problem, presents significant challenges characterized by discrete search spaces and computationally expensive performance evaluations. Standard evolutionary metaheuristics frequently encounter difficulties in such landscapes due to inefficient exploration-exploitation trade-offs and a lack of data-driven guidance, leading to excessive computational resource consumption. To address these issues, this study proposes a Synergistic Co-evolutionary Ivy Algorithm (SC-IVYA), an optimization framework specifically designed for computationally expensive black-box optimization. The proposed algorithm integrates three synergistic mechanisms to enhance search performance. First, a dynamic fusion ranking mechanism is introduced to mitigate premature convergence by incorporating genotypic diversity metrics into the selection pressure. Second, a multi-population co-evolutionary framework is designed to decouple the search process into specialized exploration and exploitation sub-tasks, facilitating global search through collaborative knowledge transfer. Finally, a surrogate-assisted predictive guidance strategy is implemented, utilizing an online radial basis function network predictor to pre-screen candidate architectures, thereby transforming the search into an informed and sample-efficient process. Empirical evaluations were conducted on both continuous mathematical benchmarks and discrete NAS tasks. On the IEEE CEC 2017 benchmark suite in high-dimensional settings, SC-IVYA exhibits highly competitive performance, achieving statistical results comparable to state-of-the-art variants such as JADE and particle swarm optimization. Further empirical validation on the NATS-Bench topology search space assesses its efficiency under strict computational budgets. Specifically, experimental results demonstrate that SC-IVYA reaches final average test accuracies of 93.92 percent, 72.45 percent, and 46.10 percent on the CIFAR-10, CIFAR-100, and ImageNet-16–120 datasets, respectively, using limited API queries. These quantitative results indicate that the proposed SC-IVYA framework improves search efficiency and provides a robust tool for solving resource-constrained engineering optimization problems.