A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes

Rusen Halepmollası; Khadija Hanifi; Ramin F. Fouladi; Ayse Tosun

doi:10.5220/0011859300003464

A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes

Rusen Halepmollası, Khadija Hanifi, Ramin F. Fouladi, Ayse Tosun

Department of Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Vulnerability prediction is a data-driven process that utilizes previous vulnerability records and their associated fixes in software development projects. Vulnerability records are rarely observed compared to other defects, even in large projects, and are usually not directly linked to the related code changes in the bug tracking system. Thus, preparing a vulnerability dataset and building a predicting model is quite challenging. There exist many studies proposing software metrics-based or embedding/token-based approaches to predict software vulnerabilities over code changes. In this study, we aim to compare the performance of two different approaches in predicting code changes that induce vulnerabilities. While the first approach is based on an aggregation of software metrics, the second approach is based on embedding representation of the source code using an Abstract Syntax Tree and skip-gram techniques. We employed Deep Learning and popular Machine Learning algorithms to predict vulnerability-inducing code changes. We report our empirical analysis over code changes on the publicly available SmartSHARK dataset that we extended by adding real vulnerability data. Software metrics-based code representation method shows a better classification performance than embedding-based code representation method in terms of recall, precision and F1-Score.

Original language	English
Title of host publication	Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023
Editors	Hermann Kaindl, Hermann Kaindl, Hermann Kaindl, Mike Mannion, Leszek Maciaszek, Leszek Maciaszek
Publisher	Science and Technology Publications, Lda
Pages	469-478
Number of pages	10
ISBN (Electronic)	9789897586477
DOIs	https://doi.org/10.5220/0011859300003464
Publication status	Published - 2023
Event	18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023 - Prague, Czech Republic Duration: 24 Apr 2023 → 25 Apr 2023

Publication series

Name	International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings
Volume	2023-April
ISSN (Electronic)	2184-4895

Conference

Conference	18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023
Country/Territory	Czech Republic
City	Prague
Period	24/04/23 → 25/04/23

Bibliographical note

Publisher Copyright:
Copyright © 2023 by SCITEPRESS - Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

Funding

This work was funded by The Scientific and Technological Research Council of Turkey, under 1515 Frontier R&D Laboratories Support Program with project no: 5169902.

Funders	Funder number
Türkiye Bilimsel ve Teknolojik Araştırma Kurumu	5169902

Keywords

Abstract Syntax Tree
Embeddings
Software Metrics
Software Vulnerabilities

Access to Document

10.5220/0011859300003464

Cite this

Halepmollası, R., Hanifi, K., Fouladi, R. F., & Tosun, A. (2023). A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes. In H. Kaindl, H. Kaindl, H. Kaindl, M. Mannion, L. Maciaszek, & L. Maciaszek (Eds.), Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023 (pp. 469-478). (International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings; Vol. 2023-April). Science and Technology Publications, Lda. https://doi.org/10.5220/0011859300003464

Halepmollası, Rusen ; Hanifi, Khadija ; Fouladi, Ramin F. et al. / A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes. Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023. editor / Hermann Kaindl ; Hermann Kaindl ; Hermann Kaindl ; Mike Mannion ; Leszek Maciaszek ; Leszek Maciaszek. Science and Technology Publications, Lda, 2023. pp. 469-478 (International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings).

@inproceedings{bb1b2bc1c7d34804894c0f7ea8fe7498,

title = "A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes",

abstract = "Vulnerability prediction is a data-driven process that utilizes previous vulnerability records and their associated fixes in software development projects. Vulnerability records are rarely observed compared to other defects, even in large projects, and are usually not directly linked to the related code changes in the bug tracking system. Thus, preparing a vulnerability dataset and building a predicting model is quite challenging. There exist many studies proposing software metrics-based or embedding/token-based approaches to predict software vulnerabilities over code changes. In this study, we aim to compare the performance of two different approaches in predicting code changes that induce vulnerabilities. While the first approach is based on an aggregation of software metrics, the second approach is based on embedding representation of the source code using an Abstract Syntax Tree and skip-gram techniques. We employed Deep Learning and popular Machine Learning algorithms to predict vulnerability-inducing code changes. We report our empirical analysis over code changes on the publicly available SmartSHARK dataset that we extended by adding real vulnerability data. Software metrics-based code representation method shows a better classification performance than embedding-based code representation method in terms of recall, precision and F1-Score.",

keywords = "Abstract Syntax Tree, Embeddings, Software Metrics, Software Vulnerabilities",

author = "Rusen Halepmollası and Khadija Hanifi and Fouladi, {Ramin F.} and Ayse Tosun",

note = "Publisher Copyright: Copyright {\textcopyright} 2023 by SCITEPRESS - Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0); 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023 ; Conference date: 24-04-2023 Through 25-04-2023",

year = "2023",

doi = "10.5220/0011859300003464",

language = "English",

series = "International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings",

publisher = "Science and Technology Publications, Lda",

pages = "469--478",

editor = "Hermann Kaindl and Hermann Kaindl and Hermann Kaindl and Mike Mannion and Leszek Maciaszek and Leszek Maciaszek",

booktitle = "Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023",

}

Halepmollası, R, Hanifi, K, Fouladi, RF & Tosun, A 2023, A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes. in H Kaindl, H Kaindl, H Kaindl, M Mannion, L Maciaszek & L Maciaszek (eds), Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023. International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings, vol. 2023-April, Science and Technology Publications, Lda, pp. 469-478, 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023, Prague, Czech Republic, 24/04/23. https://doi.org/10.5220/0011859300003464

A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes. / Halepmollası, Rusen; Hanifi, Khadija; Fouladi, Ramin F. et al.
Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023. ed. / Hermann Kaindl; Hermann Kaindl; Hermann Kaindl; Mike Mannion; Leszek Maciaszek; Leszek Maciaszek. Science and Technology Publications, Lda, 2023. p. 469-478 (International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings; Vol. 2023-April).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes

AU - Halepmollası, Rusen

AU - Hanifi, Khadija

AU - Fouladi, Ramin F.

AU - Tosun, Ayse

PY - 2023

Y1 - 2023

N2 - Vulnerability prediction is a data-driven process that utilizes previous vulnerability records and their associated fixes in software development projects. Vulnerability records are rarely observed compared to other defects, even in large projects, and are usually not directly linked to the related code changes in the bug tracking system. Thus, preparing a vulnerability dataset and building a predicting model is quite challenging. There exist many studies proposing software metrics-based or embedding/token-based approaches to predict software vulnerabilities over code changes. In this study, we aim to compare the performance of two different approaches in predicting code changes that induce vulnerabilities. While the first approach is based on an aggregation of software metrics, the second approach is based on embedding representation of the source code using an Abstract Syntax Tree and skip-gram techniques. We employed Deep Learning and popular Machine Learning algorithms to predict vulnerability-inducing code changes. We report our empirical analysis over code changes on the publicly available SmartSHARK dataset that we extended by adding real vulnerability data. Software metrics-based code representation method shows a better classification performance than embedding-based code representation method in terms of recall, precision and F1-Score.

AB - Vulnerability prediction is a data-driven process that utilizes previous vulnerability records and their associated fixes in software development projects. Vulnerability records are rarely observed compared to other defects, even in large projects, and are usually not directly linked to the related code changes in the bug tracking system. Thus, preparing a vulnerability dataset and building a predicting model is quite challenging. There exist many studies proposing software metrics-based or embedding/token-based approaches to predict software vulnerabilities over code changes. In this study, we aim to compare the performance of two different approaches in predicting code changes that induce vulnerabilities. While the first approach is based on an aggregation of software metrics, the second approach is based on embedding representation of the source code using an Abstract Syntax Tree and skip-gram techniques. We employed Deep Learning and popular Machine Learning algorithms to predict vulnerability-inducing code changes. We report our empirical analysis over code changes on the publicly available SmartSHARK dataset that we extended by adding real vulnerability data. Software metrics-based code representation method shows a better classification performance than embedding-based code representation method in terms of recall, precision and F1-Score.

KW - Abstract Syntax Tree

KW - Embeddings

KW - Software Metrics

KW - Software Vulnerabilities

UR - http://www.scopus.com/inward/record.url?scp=85150987535&partnerID=8YFLogxK

U2 - 10.5220/0011859300003464

DO - 10.5220/0011859300003464

M3 - Conference contribution

AN - SCOPUS:85150987535

T3 - International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings

SP - 469

EP - 478

BT - Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023

A2 - Kaindl, Hermann

A2 - Mannion, Mike

A2 - Maciaszek, Leszek

PB - Science and Technology Publications, Lda

T2 - 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023

Y2 - 24 April 2023 through 25 April 2023

ER -

Halepmollası R, Hanifi K, Fouladi RF, Tosun A. A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes. In Kaindl H, Kaindl H, Kaindl H, Mannion M, Maciaszek L, Maciaszek L, editors, Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE 2023. Science and Technology Publications, Lda. 2023. p. 469-478. (International Conference on Evaluation of Novel Approaches to Software Engineering, ENASE - Proceedings). doi: 10.5220/0011859300003464

A Comparison of Source Code Representation Methods to Predict Vulnerability Inducing Code Changes

Abstract

Publication series

Conference

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this