Experimentation in Software Engineering

by Claes Wohlin, Per Runeson, Martin Höst

This textbook, which has become the leading source for empirical software engineering research through its earlier editions, provides an in-depth introduction to experimentation in software engineering. The focus is on the steps to go through when conducting experiments. It also introduces a structure for selecting a research design and an introduction to systematic literature reviews, surveys, and case study research. Experiments are valuable tools for all software engineers who evaluate and choose between different solutions, methods, techniques, languages, and tools.



The book has three main parts. Part I introduces empirical research through a structure for selecting a research design and explains several essential areas related to empirical research. Moreover, it provides introductions to systematic literature reviews, experiments, surveys, and case studies. Part II devotes one chapter to each of the five experiment scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendices. The book provides indispensable information regarding empirical studies, particularly for experiments, but also for systematic literature studies, surveys, and case studies. This new edition comprises substantial new material. This includes a chapter on selecting a research design, which helps put experimentation into a broader context of other research approaches. Furthermore, a new chapter on survey research has been added. Novel contributions concerning A/B testing, replications, open science, validity threats, paired comparison design and tools for statistical analysis have been added. Moreover, systematic literature studies, experiments, and case study research descriptions have been consolidated. The book has also been updated based on the evolution of software engineering since the previous edition's publication.



The book is intended for students, teachers, researchers, and practitioners who would like to compare different options, for example, feature designs, or different development methods, techniques, languages, or tools.  Teachers can use the book for undergraduate or graduate students; the book can be used as a course book on experimentation and empirical studies in general. Researchers will also benefit from the book by learning more about how to conduct empirical studies. Likewise, practitioners may use it as a “cookbook” when evaluating different solutions, alternatively for evaluating new methods or techniques before implementing them in their organisation.

Genres: Computer Science, Reference

522 pages, Kindle Edition

First published December 31, 1999

Reviews

[Rares Cristea · Rating 3 out of 5]

So I've mostly skimmed this book, and emphasised on reading the first couple of chapters.

This is a very in depth look at the formal methods needed to apply to create a successful Software Engineering Experiment. I've read it as part of my dissertation thesis, and could concur it helped me a bit. It is very technical, even for what i deem necessary for just a dissertation. However, as it tries to cover a topic not only niche, but one of the few niches that is less formal in one of the most formalised scientific domains. The volatility of best practices in the subject's domain, also doesn't help this book age well. Sure, the agile practices described here are still in practice as they were mostly when the book was written. But it fails to properly address integrating Cloud services and modern CI/CD in the experimentation framework.

Overall, a great book to skim for an introduction on how to formalise something that it is not quite made for formal research.

[Rodrigo Lima · Rating 5 out of 5]

Best book for software engineering experimentation