Evolutionary Training of Neural Networks: The Role of Crossover Operators in Genetic Algorithms Compared with Backpropagation

Abstract

Training neural networks with gradient-based methods such as backpropagation is the dominant paradigm, but it depends on differentiable loss functions and is sensitive to initialization and local minima. Evolutionary algorithms offer a gradient-free alternative, yet the influence of their internal operators on training quality remains insufficiently characterized. This study presents a systematic comparison of backpropagation and ten variants of a genetic algorithm (GA) for training multi-layer perceptrons (MLPs), with particular focus on the role of crossover operators. The evaluation covers four MLP architectures and ten classification datasets from the UCI Machine Learning Repository, differing in sample size, dimensionality, and number of classes. Each configuration was assessed using stratified 4-fold cross-validation with 30 independent repetitions, and accuracy served as the primary performance metric, with macro-F1 reported to assess classifier behavior on class-imbalanced datasets. Backpropagation achieved higher mean accuracy than every GA variant on nine of the ten datasets, with the largest margins on high-dimensional problems. The genetic algorithm proved competitive on simpler, class-balanced datasets, where its better-performing variants matched the gradient-based baseline within one to two percentage points, and, on the Heart disease dataset, every GA variant reached a higher mean accuracy than backpropagation across all four architectures, though absolute performance remained modest on this five-class problem. Among crossover operators, BLX-α and BLX-α-β combined with tournament selection and a high crossover probability yielded the strongest configurations, while averaging crossover performed worst, as it restricts offspring to the midpoint of the parents and cannot explore beyond the range already present in the population. Tournament selection consistently led to higher mean accuracy than roulette-wheel selection, and shallow but moderately wide architectures, which encode fewer trainable parameters and thus a shorter chromosome, proved more amenable to evolutionary training than the two-layer alternative. These findings clarify when gradient-free training is competitive and which evolutionary operators drive its effectiveness.