Are students representatives of professionals in software engineering experiments?

Iflaah Salman, Ayse Tosun Misirli, Natalia Juristo

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

260 Citations (Scopus)

Abstract

Background: Most of the experiments in software engineering (SE) employ students as subjects. This raises concerns about the realism of the results acquired through students and adaptability of the results to software industry. Aim: We compare students and professionals to understand how well students represent professionals as experimental subjects in SE research. Method: The comparison was made in the context of two test-driven development experiments conducted with students in an academic setting and with professionals in a software organization. We measured the code quality of several tasks implemented by both subject groups and checked whether students and professionals perform similarly in terms of code quality metrics. Results: Except for minor differences, neither of the subject groups is better than the other. Professionals produce larger, yet less complex, methods when they use their traditional development approach, whereas both subject groups perform similarly when they apply a new approach for the first time. Conclusion: Given a carefully scoped experiment on a development approach that is new to both students and professionals, similar performances are observed. Further investigation is necessary to analyze the effects of subject demographics and level of experience on the results of SE experiments.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, ICSE 2015
PublisherIEEE Computer Society
Pages666-676
Number of pages11
ISBN (Electronic)9781479919345
DOIs
Publication statusPublished - 12 Aug 2015
Event37th IEEE/ACM International Conference on Software Engineering, ICSE 2015 - Florence, Italy
Duration: 16 May 201524 May 2015

Publication series

NameProceedings - International Conference on Software Engineering
Volume1
ISSN (Print)0270-5257

Conference

Conference37th IEEE/ACM International Conference on Software Engineering, ICSE 2015
Country/TerritoryItaly
CityFlorence
Period16/05/1524/05/15

Bibliographical note

Publisher Copyright:
© 2015 IEEE.

Keywords

  • Code quality
  • Empirical study
  • Experimentation
  • Test-driven development

Fingerprint

Dive into the research topics of 'Are students representatives of professionals in software engineering experiments?'. Together they form a unique fingerprint.

Cite this