Browsing by Author "Brereton, P."
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Open Access The contribution that empirical studies performed in industry make to the findings of systematic reviews: a tertiary study(2017) Budgen, D.; Brereton, P.; Williams, N.; Drummond, S.Context Systematic reviews can provide useful knowledge for software engineering practice, by aggregating and synthesising empirical studies related to a specific topic. Objective We sought to assess how far the findings of systematic reviews addressing practice-oriented topics have been derived from empirical studies that were performed in industry or that used industry data. Method We drew upon and augmented the data obtained from a tertiary study that performed a systematic review of systematic reviews published in the period up to the end of 2015, seeking to identify those with findings that are relevant for teaching and practice. For the supplementary analysis reported here, we then examined the profiles of the primary studies as reported in each systematic review. Results We identified 48 systematic reviews as candidates for further analysis. The many differences that arise between systematic reviews, together with the incompleteness of reporting for these, mean that our counts should be treated as indicative rather than definitive. However, even when allowing for problems of classification, the findings from the majority of these systematic reviews were predominantly derived from using primary studies conducted in industry. There was also an emphasis upon the use of case studies, and a number of the systematic reviews also made some use of weaker ‘experience’ or even ‘opinion’ papers. Conclusions Primary studies from industry play an important role as inputs to systematic reviews. Using more rigorous industry-based primary studies can give greater authority to the findings of the systematic reviews, and should help with the creation of a corpus of sound empirical data to support evidence-informed decisions.Item Open Access Reporting systematic reviews: Some lessons from a tertiary study(Elsevier, 2017-10-27) Budgen, D.; Brereton, P.; Drummond, S.; Williams, N.Context Many of the systematic reviews published in software engineering are related to research or methodological issues and hence are unlikely to be of direct benefit to practitioners or teachers. Those that are relevant to practice and teaching need to be presented in a form that makes their findings usable with minimum interpretation. Objective We have examined a sample of the many systematic reviews that have been published over a period of six years, in order to assess how well these are reported and identify useful lessons about how this might be done. Method We undertook a tertiary study, performing a systematic review of systematic reviews. Our study found 178 systematic reviews published in a set of major software engineering journals over the period 2010–2015. Of these, 37 provided recommendations or conclusions of relevance to education and/or practice and we used the DARE criteria as well as other attributes related to the systematic review process to analyse how well they were reported. Results We have derived a set of 12 ‘lessons’ that could help authors with reporting the outcomes of a systematic review in software engineering. We also provide an associated checklist for use by journal and conference referees. Conclusion There are several areas where better reporting is needed, including quality assessment, synthesis, and the procedures followed by the reviewers. Researchers, practitioners, teachers and journal referees would all benefit from better reporting of systematic reviews, both for clarity and also for establishing the provenance of any findings.