HAR-MLP: A Hybrid Attention–Residual MLP Architecture with Handcrafted Features for Software Bug Prediction

Abstract

: Open-source platforms and issue tracking systems such as GitHub and Jira generate large volumes of issue reports and code changes, making effective bug identification a challenging task. This study investigates software bug prediction by integrating various feature extraction methods, including Word2Vec, TF-IDF, FastText, GloVe, and Doc2Vec, with several lightweight ML algorithms. Hybrid feature sets are further enhanced using Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) and empirical results indicate that Word2Vec and Multi-Layer Perceptron (MLP) provide comparatively stronger performance. The study proposes a Hybrid Attention-Residual Multilayer Perceptron (HAR-MLP) model to automatically classify software issue reports as bugs and feature requests. The model is equipped with a feature set consisting of a combination of TF-IDF reweighted embedding averages derived from Word2Vec embedding vectors, mean-maximum-standard deviation pooling statistics, and DCT spectral coefficients. This feature fusion is then combined with context-sensitive representations obtained from a multi-headed self-attention mechanism and fed as input to the Residual MLP model as a classifier. While traditional MLP has direct forward connections between layers, Residual MLP reduces gradient decay and prevents information loss in deep layers by combining intermediate layer outputs with the input using the skip connection method. This structure increases the model’s learning capacity and provides higher accuracy, especially in hybrid structures where attention outputs and handcrafted features are processed together. Experimental results demonstrate that the HAR-MLP model achieves higher performance than traditional feature extraction and classification methods while remaining computationally competitive compared to pre-trained transformer-based approaches. Moreover, cross-platform evaluation on three independent Jira issue datasets further demonstrates the generalizability of the proposed architecture, achieving macro-F1 scores ranging from 86.61% to 97.48%. The source code of the proposed HAR-MLP model can be accessed via the following link: https://github.com/isilkarabeyaksakalli/HAR-MLP.