Defect prediction on a legacy industrial software: A case study on software with few defects

Context: Building defect prediction models for software projects is helpful for reducing the effort in locating defects. In this paper, we share our experiences in building a defect prediction model for a large industrial software project. We extract product and process metrics to build models and show that we can build an accurate defect prediction model even when 4% of the software is defective. Objective: Our goal in this project is to integrate a defect predictor into the continuous integration (CI) cycle of a large software project and decrease the effort in testing. Method: We present our approach in the form of an experience report. Specifically, we collected data from seven older versions of the software project and used additional features to predict defects of current versions. We compared several classification techniques including Naive Bayes, Decision Trees, and Random Forest and resampled our training data to present the company with the most accurate defect predictor. Results: Our results indicate that we can focus testing efforts by guiding the test team to only 8% of the software where 53% of actual defects can be found. Our model has 90% accuracy. Conclusion: We produce a defect prediction model with high accuracy for a software with defect rate of 4%. Our model uses Random Forest, that which we show has more predictive power than Naive Bayes, Logistic Regression and Decision Trees in our case.