Predicting defects using test execution logs in an industrial setting

Ayse Tosun; Ozgur Turkgulu; Dogan Razon; Hamza Yusuf Aydemir; Arda Gureller

doi:10.1109/ICSE-C.2017.148

Predicting defects using test execution logs in an industrial setting

Ayse Tosun, Ozgur Turkgulu, Dogan Razon, Hamza Yusuf Aydemir, Arda Gureller

Department of Computer Engineering

Ericsson AB

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Researchers often focus on the development process and the final product (source code) to investigate and predict software defects. Unfortunately, these models may not be applicable to software projects in which there is no access to the data sources regarding development process. For example, in cases when a company conducts tests on behalf of its business contractors, it is only possible to evaluate in-process quality of the company based on its testing process. We present an industrial case at Ericsson Turkey that illustrates such a business constraint. We define a set of in-process testing metrics that are extracted from acceptance test execution logs of a large scale software application developed at Ericsson Turkey. We measure the acceptance testing process of 15 weeks using these metrics, and predict the number of defects reported in weekly acceptance tests. We report our measurement, model construction and assessment steps in this paper.

Original language	English
Title of host publication	Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	294-296
Number of pages	3
ISBN (Electronic)	9781538615898
DOIs	https://doi.org/10.1109/ICSE-C.2017.148
Publication status	Published - 30 Jun 2017
Event	39th IEEE/ACM International Conference on Software Engineering Companion, ICSE-C 2017 - Buenos Aires, Argentina Duration: 20 May 2017 → 28 May 2017

Publication series

Name	Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017

Conference

Conference	39th IEEE/ACM International Conference on Software Engineering Companion, ICSE-C 2017
Country/Territory	Argentina
City	Buenos Aires
Period	20/05/17 → 28/05/17

Bibliographical note

Publisher Copyright:
© 2017 IEEE.

Keywords

Acceptance testing
Industrial case study
Software defect prediction
Test execution logs

Access to Document

10.1109/ICSE-C.2017.148

Cite this

Tosun, A., Turkgulu, O., Razon, D., Aydemir, H. Y., & Gureller, A. (2017). Predicting defects using test execution logs in an industrial setting. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017 (pp. 294-296). Article 7965335 (Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICSE-C.2017.148

Tosun, Ayse ; Turkgulu, Ozgur ; Razon, Dogan et al. / Predicting defects using test execution logs in an industrial setting. Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 294-296 (Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017).

@inproceedings{63c9d51dd3374eeea48a4b795fc58ad9,

title = "Predicting defects using test execution logs in an industrial setting",

abstract = "Researchers often focus on the development process and the final product (source code) to investigate and predict software defects. Unfortunately, these models may not be applicable to software projects in which there is no access to the data sources regarding development process. For example, in cases when a company conducts tests on behalf of its business contractors, it is only possible to evaluate in-process quality of the company based on its testing process. We present an industrial case at Ericsson Turkey that illustrates such a business constraint. We define a set of in-process testing metrics that are extracted from acceptance test execution logs of a large scale software application developed at Ericsson Turkey. We measure the acceptance testing process of 15 weeks using these metrics, and predict the number of defects reported in weekly acceptance tests. We report our measurement, model construction and assessment steps in this paper.",

keywords = "Acceptance testing, Industrial case study, Software defect prediction, Test execution logs",

author = "Ayse Tosun and Ozgur Turkgulu and Dogan Razon and Aydemir, {Hamza Yusuf} and Arda Gureller",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 39th IEEE/ACM International Conference on Software Engineering Companion, ICSE-C 2017 ; Conference date: 20-05-2017 Through 28-05-2017",

year = "2017",

month = jun,

day = "30",

doi = "10.1109/ICSE-C.2017.148",

language = "English",

series = "Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "294--296",

booktitle = "Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017",

address = "United States",

}

Tosun, A, Turkgulu, O, Razon, D, Aydemir, HY & Gureller, A 2017, Predicting defects using test execution logs in an industrial setting. in Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017., 7965335, Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017, Institute of Electrical and Electronics Engineers Inc., pp. 294-296, 39th IEEE/ACM International Conference on Software Engineering Companion, ICSE-C 2017, Buenos Aires, Argentina, 20/05/17. https://doi.org/10.1109/ICSE-C.2017.148

Predicting defects using test execution logs in an industrial setting. / Tosun, Ayse; Turkgulu, Ozgur; Razon, Dogan et al.
Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 294-296 7965335 (Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Predicting defects using test execution logs in an industrial setting

AU - Tosun, Ayse

AU - Turkgulu, Ozgur

AU - Razon, Dogan

AU - Aydemir, Hamza Yusuf

AU - Gureller, Arda

PY - 2017/6/30

Y1 - 2017/6/30

N2 - Researchers often focus on the development process and the final product (source code) to investigate and predict software defects. Unfortunately, these models may not be applicable to software projects in which there is no access to the data sources regarding development process. For example, in cases when a company conducts tests on behalf of its business contractors, it is only possible to evaluate in-process quality of the company based on its testing process. We present an industrial case at Ericsson Turkey that illustrates such a business constraint. We define a set of in-process testing metrics that are extracted from acceptance test execution logs of a large scale software application developed at Ericsson Turkey. We measure the acceptance testing process of 15 weeks using these metrics, and predict the number of defects reported in weekly acceptance tests. We report our measurement, model construction and assessment steps in this paper.

AB - Researchers often focus on the development process and the final product (source code) to investigate and predict software defects. Unfortunately, these models may not be applicable to software projects in which there is no access to the data sources regarding development process. For example, in cases when a company conducts tests on behalf of its business contractors, it is only possible to evaluate in-process quality of the company based on its testing process. We present an industrial case at Ericsson Turkey that illustrates such a business constraint. We define a set of in-process testing metrics that are extracted from acceptance test execution logs of a large scale software application developed at Ericsson Turkey. We measure the acceptance testing process of 15 weeks using these metrics, and predict the number of defects reported in weekly acceptance tests. We report our measurement, model construction and assessment steps in this paper.

KW - Acceptance testing

KW - Industrial case study

KW - Software defect prediction

KW - Test execution logs

UR - http://www.scopus.com/inward/record.url?scp=85026787321&partnerID=8YFLogxK

U2 - 10.1109/ICSE-C.2017.148

DO - 10.1109/ICSE-C.2017.148

M3 - Conference contribution

AN - SCOPUS:85026787321

T3 - Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017

SP - 294

EP - 296

BT - Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 39th IEEE/ACM International Conference on Software Engineering Companion, ICSE-C 2017

Y2 - 20 May 2017 through 28 May 2017

ER -

Tosun A, Turkgulu O, Razon D, Aydemir HY, Gureller A. Predicting defects using test execution logs in an industrial setting. In Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 294-296. 7965335. (Proceedings - 2017 IEEE/ACM 39th International Conference on Software Engineering Companion, ICSE-C 2017). doi: 10.1109/ICSE-C.2017.148

Predicting defects using test execution logs in an industrial setting

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this