Abstract
Background: Most of the experiments in software engineering (SE) employ students as subjects. This raises concerns about the realism of the results acquired through students and adaptability of the results to software industry. Aim: We compare students and professionals to understand how well students represent professionals as experimental subjects in SE research. Method: The comparison was made in the context of two test-driven development experiments conducted with students in an academic setting and with professionals in a software organization. We measured the code quality of several tasks implemented by both subject groups and checked whether students and professionals perform similarly in terms of code quality metrics. Results: Except for minor differences, neither of the subject groups is better than the other. Professionals produce larger, yet less complex, methods when they use their traditional development approach, whereas both subject groups perform similarly when they apply a new approach for the first time. Conclusion: Given a carefully scoped experiment on a development approach that is new to both students and professionals, similar performances are observed. Further investigation is necessary to analyze the effects of subject demographics and level of experience on the results of SE experiments.
Original language | English |
---|---|
Title of host publication | Proceedings - 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, ICSE 2015 |
Publisher | IEEE Computer Society |
Pages | 666-676 |
Number of pages | 11 |
ISBN (Electronic) | 9781479919345 |
DOIs | |
Publication status | Published - 12 Aug 2015 |
Event | 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015 - Florence, Italy Duration: 16 May 2015 → 24 May 2015 |
Publication series
Name | Proceedings - International Conference on Software Engineering |
---|---|
Volume | 1 |
ISSN (Print) | 0270-5257 |
Conference
Conference | 37th IEEE/ACM International Conference on Software Engineering, ICSE 2015 |
---|---|
Country/Territory | Italy |
City | Florence |
Period | 16/05/15 → 24/05/15 |
Bibliographical note
Publisher Copyright:© 2015 IEEE.
Keywords
- Code quality
- Empirical study
- Experimentation
- Test-driven development