Towards an operationalization of test-driven development skills: An industrial empirical study

Davide Fucci*, Burak Turhan, Natalia Juristo, Oscar Dieste, Ayse Tosun-Misirli, Markku Oivo

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


Context: The majority of the empirical studies on Test-driven development (TDD) are concerned with verifying or refuting the effectiveness of the technique over a traditional approach, and they tend to neglect whether the subjects possess the necessary skills to apply TDD, though they argue such skills are necessary. Objective: We evaluate a set of minimal, a priori and in process skills necessary to apply TDD. We determine whether variations in external quality (i.e., number of defects) and productivity (i.e., number of features implemented) can be associated with different clusters of the TDD skills' set. Method: We executed a quasi-experiment involving 30 practitioners from industry. We first grouped the participants according to their TDD skills' set (consisting of a priori experience on programming and testing as well as in-process TDD conformance) into three levels (Low-Medium-High) using k-means clustering. We then applied ANOVA to compare the clusters in terms of external quality and productivity, and conducted post-hoc pairwise analysis. Results: We did not observe a statistically significant difference between the clusters either for external software quality (F(2,27=1.44,p=.260), or productivity (F(2,27)=3.02,p=.065). However, the analysis of the effect sizes and their confidence intervals shows that the TDD skills' set is a factor that could account for up to 28% of the external quality, and 38% for productivity. Conclusion: We have reason to conclude that focusing on the improvement of TDD skills' set investigated in this study could benefit software developers in improving their baseline productivity and the external quality of the code they produce. However, replications are needed to overcome the issues related with the statistical power of this study. We suggest practical insights for future work to investigate the phenomenon further.

Original languageEnglish
Pages (from-to)82-97
Number of pages16
JournalInformation and Software Technology
Publication statusPublished - 1 Dec 2015

Bibliographical note

Publisher Copyright:
© 2015 Elsevier B.V. All rights reserved.


This research is partially supported by the Academy of Finland with decision no.: 278354 , and by Finnish Distinguished Professor (Fi.Di.Pro.) programme, ESEIL. The first author would like to acknowledge the Nokia Foundation and ISACA Finland chapter for the support provided in completing this work. We would like to acknowledge Dr. Lucas Layman who significantly contributed to the design of the task used in the study. We would like to acknowledge the anonymous reviewers for their helpful comments and detailed suggestions.

FundersFunder number
Academy of Finland278354
Nokia Foundation


    • Developers' productivity
    • Process conformance
    • Software quality
    • Test-driven development


    Dive into the research topics of 'Towards an operationalization of test-driven development skills: An industrial empirical study'. Together they form a unique fingerprint.

    Cite this