Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study

Oscar Dieste; Alejandrina M. Aranda; Fernando Uyaguari; Burak Turhan; Ayse Tosun; Davide Fucci; Markku Oivo; Natalia Juristo

doi:10.1007/s10664-016-9471-3

Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study

Oscar Dieste^*, Alejandrina M. Aranda, Fernando Uyaguari, Burak Turhan, Ayse Tosun, Davide Fucci, Markku Oivo, Natalia Juristo

^*Corresponding author for this work

Department of Computer Engineering

Research output: Contribution to journal › Article › peer-review

39 Citations (Scopus)

Abstract

Context: There is a widespread belief in both SE and other branches of science that experience helps professionals to improve their performance. However, cases have been reported where experience not only does not have a positive influence but sometimes even degrades the performance of professionals. Aim: Determine whether years of experience influence programmer performance. Method: We have analysed 10 quasi-experiments executed both in academia with graduate and postgraduate students and in industry with professionals. The experimental task was to apply ITLD on two experimental problems and then measure external code quality and programmer productivity. Results: Programming experience gained in industry does not appear to have any effect whatsoever on quality and productivity. Overall programming experience gained in academia does tend to have a positive influence on programmer performance. These two findings may be related to the fact that, as opposed to deliberate practice, routine practice does not appear to lead to improved performance. Experience in the use of productivity tools, such as testing frameworks and IDE also has positive effects. Conclusion: Years of experience are a poor predictor of programmer performance. Academic background and specialized knowledge of task-related aspects appear to be rather good predictors.

Original language	English
Pages (from-to)	2457-2542
Number of pages	86
Journal	Empirical Software Engineering
Volume	22
Issue number	5
DOIs	https://doi.org/10.1007/s10664-016-9471-3
Publication status	Published - 1 Oct 2017

Bibliographical note

Publisher Copyright:
© 2017, Springer Science+Business Media New York.

Funding

We conducted 10 quasi-experiments, six of which were run in industry and four in academia. All the quasi-experiments were conducted as part of the Empirical Software Engineering Industry Lab (ESEIL) project, led by N. Juristo and funded by TEKES. The research has been conducted according to the regulations laid out by the Universidad Politécnica de Madrid and University of Oulu’s Ethical Boards. Both the funding agency and the participating researchers state that they have no conflicts of interest with respect to the research results. In all cases, the experimental procedure was as follows:

Funders	Funder number
Tekes

Keywords

Academy
Experience
External quality
Industry
Iterative test-last development
Performance
Productivity
Programming

Access to Document

10.1007/s10664-016-9471-3

Cite this

@article{1aeea849990243a180d2f91cfb3c4706,

title = "Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study",

abstract = "Context: There is a widespread belief in both SE and other branches of science that experience helps professionals to improve their performance. However, cases have been reported where experience not only does not have a positive influence but sometimes even degrades the performance of professionals. Aim: Determine whether years of experience influence programmer performance. Method: We have analysed 10 quasi-experiments executed both in academia with graduate and postgraduate students and in industry with professionals. The experimental task was to apply ITLD on two experimental problems and then measure external code quality and programmer productivity. Results: Programming experience gained in industry does not appear to have any effect whatsoever on quality and productivity. Overall programming experience gained in academia does tend to have a positive influence on programmer performance. These two findings may be related to the fact that, as opposed to deliberate practice, routine practice does not appear to lead to improved performance. Experience in the use of productivity tools, such as testing frameworks and IDE also has positive effects. Conclusion: Years of experience are a poor predictor of programmer performance. Academic background and specialized knowledge of task-related aspects appear to be rather good predictors.",

keywords = "Academy, Experience, External quality, Industry, Iterative test-last development, Performance, Productivity, Programming",

author = "Oscar Dieste and Aranda, {Alejandrina M.} and Fernando Uyaguari and Burak Turhan and Ayse Tosun and Davide Fucci and Markku Oivo and Natalia Juristo",

note = "Publisher Copyright: {\textcopyright} 2017, Springer Science+Business Media New York.",

year = "2017",

month = oct,

day = "1",

doi = "10.1007/s10664-016-9471-3",

language = "English",

volume = "22",

pages = "2457--2542",

journal = "Empirical Software Engineering",

issn = "1382-3256",

publisher = "Springer Netherlands",

number = "5",

}

TY - JOUR

T1 - Empirical evaluation of the effects of experience on code quality and programmer productivity

T2 - an exploratory study

AU - Dieste, Oscar

AU - Aranda, Alejandrina M.

AU - Uyaguari, Fernando

AU - Turhan, Burak

AU - Tosun, Ayse

AU - Fucci, Davide

AU - Oivo, Markku

AU - Juristo, Natalia

PY - 2017/10/1

Y1 - 2017/10/1

N2 - Context: There is a widespread belief in both SE and other branches of science that experience helps professionals to improve their performance. However, cases have been reported where experience not only does not have a positive influence but sometimes even degrades the performance of professionals. Aim: Determine whether years of experience influence programmer performance. Method: We have analysed 10 quasi-experiments executed both in academia with graduate and postgraduate students and in industry with professionals. The experimental task was to apply ITLD on two experimental problems and then measure external code quality and programmer productivity. Results: Programming experience gained in industry does not appear to have any effect whatsoever on quality and productivity. Overall programming experience gained in academia does tend to have a positive influence on programmer performance. These two findings may be related to the fact that, as opposed to deliberate practice, routine practice does not appear to lead to improved performance. Experience in the use of productivity tools, such as testing frameworks and IDE also has positive effects. Conclusion: Years of experience are a poor predictor of programmer performance. Academic background and specialized knowledge of task-related aspects appear to be rather good predictors.

AB - Context: There is a widespread belief in both SE and other branches of science that experience helps professionals to improve their performance. However, cases have been reported where experience not only does not have a positive influence but sometimes even degrades the performance of professionals. Aim: Determine whether years of experience influence programmer performance. Method: We have analysed 10 quasi-experiments executed both in academia with graduate and postgraduate students and in industry with professionals. The experimental task was to apply ITLD on two experimental problems and then measure external code quality and programmer productivity. Results: Programming experience gained in industry does not appear to have any effect whatsoever on quality and productivity. Overall programming experience gained in academia does tend to have a positive influence on programmer performance. These two findings may be related to the fact that, as opposed to deliberate practice, routine practice does not appear to lead to improved performance. Experience in the use of productivity tools, such as testing frameworks and IDE also has positive effects. Conclusion: Years of experience are a poor predictor of programmer performance. Academic background and specialized knowledge of task-related aspects appear to be rather good predictors.

KW - Academy

KW - Experience

KW - External quality

KW - Industry

KW - Iterative test-last development

KW - Performance

KW - Productivity

KW - Programming

UR - http://www.scopus.com/inward/record.url?scp=85011654161&partnerID=8YFLogxK

U2 - 10.1007/s10664-016-9471-3

DO - 10.1007/s10664-016-9471-3

M3 - Article

AN - SCOPUS:85011654161

SN - 1382-3256

VL - 22

SP - 2457

EP - 2542

JO - Empirical Software Engineering

JF - Empirical Software Engineering

IS - 5

ER -

Empirical evaluation of the effects of experience on code quality and programmer productivity: an exploratory study

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this