Deep Learning-based Software Vulnerability Severity Prediction Using Natural Language Processing

Abstract

Software vulnerability assessment is essential for prioritising remediation, yet the manual assignment of Common Vulnerability Scoring System metrics is time-consuming, subjective, and prone to error. This study develops and evaluates a design science artefact for predicting vulnerability severity directly from natural-language descriptions. More than 15,000 confirmed vulnerability records were collected from ten open-source projects represented in the CVE-NVD repositories. The descriptions were cleaned through punctuation removal, tokenisation, stopword removal, and Porter stemming. Textual features were represented using n-grams, term frequency-inverse document frequency, and Word2Vec, while Information Gain was applied for feature selection. A deep neural network with three hidden layers was developed for binary classification of vulnerabilities as Severe or Non-Severe. Its performance was compared with Random Forest, Support Vector Machine, Decision Tree, k-nearest neighbours, and Naïve Bayes classifiers using stratified 10-fold cross-validation. The deep neural network achieved average accuracy, precision, and recall values of 71.2%, 68.5%, and 72.0%, respectively, and generally outperformed the conventional classifiers across the reported metrics. However, performance varied among projects, with AUC-ROC values ranging from 0.385 for Gentoo to 0.720 for Windows 7, and the Decision Tree exceeded the deep neural network on the Gentoo dataset. These findings indicate that natural-language descriptions can support automated severity prediction, while also showing that model performance depends on project-specific data characteristics and data availability.