Field Studies: A methodology for construction and evaluation of recommendation systems in software engineering

One way to implement and evaluate the effectiveness of recommendation systems in software engineering is to conduct field studies. Field studies are important as they are the extension of laboratory experiments into real-life situations of organizations and/or society. They bring greater realism to the phenomena that are under study. However, field studies require following a rigorous research approach with many challenges attached, such as difficulties in implementing the research design, achieving sufficient control, replication, validity, and reliability. In practice, another challenge is to find organizations who are prepared to be studied. In this chapter, we provide a step-by-step process for the construction and deployment of recommendation systems in software engineering in the field. We also emphasize three main challenges (organizational, data, design) encountered during field studies, both in general and specifically with respect to software organizations.