A conceptual replication on predicting the severity of software vulnerabilities

Software vulnerabilities may lead to crucial security risks in software systems. Thus, prioritization of the vulnerabilities is an important task for security teams, and assessing how severe the vulnerabilities are would help teams during fixing and maintenance activities. We replicated a prior work which aims to predict the severity of software vulnerabilities by grouping vulnerabilities into different severity levels. We follow their approach on feature extraction using word embeddings, and on prediction model using Convolutional Neural Networks (CNNs). In addition, Long Short Term Memory (LSTM) and Extreme Gradient Boosting (XGBoost) models are used. We also extend the replicated work by aiming to predict severity scores rather than levels. We carried out two experiments for predicting severity levels and severity scores of 82,974 vulnerabilities. On predicting the severity levels, our LSTM and CNN models perform similarly with an F1 score of 0.756 F1 score and 0.752, respectively. On predicting the severity scores, LSTM, CNN and XGBoost models perform 16.14%, 17.03%, 18.91% MAPE values, respectively.