Predicting defects using test execution logs in an industrial setting

Researchers often focus on the development process and the final product (source code) to investigate and predict software defects. Unfortunately, these models may not be applicable to software projects in which there is no access to the data sources regarding development process. For example, in cases when a company conducts tests on behalf of its business contractors, it is only possible to evaluate in-process quality of the company based on its testing process. We present an industrial case at Ericsson Turkey that illustrates such a business constraint. We define a set of in-process testing metrics that are extracted from acceptance test execution logs of a large scale software application developed at Ericsson Turkey. We measure the acceptance testing process of 15 weeks using these metrics, and predict the number of defects reported in weekly acceptance tests. We report our measurement, model construction and assessment steps in this paper.