What do you think?
Rate this book
The Leprechauns of Software Engineering
The software profession has a problem, widely recognized but which nobody seems willing to do anything about. You can think of this problem as a variant of the well known "telephone game", where some trivial rumor is repeated from one person to the next until it has become distorted beyond recognition and blown up out of all proportion.
Unfortunately, the objects of this telephone game are generally considered cornerstone truths of the discipline, to the point that their acceptance now seems to hinder further progress.
In this short ebook, we will take a look at some of those "ground truths": the claimed 10x variation in productivity between developers; the "software crisis"; the cost-of-change curve; the "cone of uncertainty"; and more. We'll hone our scholarship skills by looking up the original source for these ideas and taking a deep dive in the history of their development. We'll assess the real weight of the evidence behind these ideas.
And we'll confront the scary prospect of moving the state of the art forward in a discipline that has had the ground kicked from under it.
Unfortunately, the objects of this telephone game are generally considered cornerstone truths of the discipline, to the point that their acceptance now seems to hinder further progress.
In this short ebook, we will take a look at some of those "ground truths": the claimed 10x variation in productivity between developers; the "software crisis"; the cost-of-change curve; the "cone of uncertainty"; and more. We'll hone our scholarship skills by looking up the original source for these ideas and taking a deep dive in the history of their development. We'll assess the real weight of the evidence behind these ideas.
And we'll confront the scary prospect of moving the state of the art forward in a discipline that has had the ground kicked from under it.
183 pages, ebook
First published January 1, 2012
9 people are currently reading
701 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 31 reviews
March 29, 2017
'Software engineering' tries hard to be a 'real science'. In the last four decades, some claims have gained general acceptance: 10x developers exist, and defects become more costly to fix the later they're discovered. These claims are often followed by a list of imposing citations.
In The Leprechauns of Software Engineering, Laurent Bossavit chases down the sources of these accepted 'facts', and in the process demonstrates that there's no science behind them.
While he explains, in detail, how he manages to unravel the various citations in order to get to the heart of the matter, the book is easy to read. At times, it's like reading a thriller; I was literally turning pages, and didn't want to put the book down.
There's a risk that I suffer from confirmation bias, but this could be one of the decade's most important books about software development.
My only problem with the book is that I feel that I should not take citations for granted. Instead, in its spirit, I should get to the original sources, in order to verify that the author hasn't misrepresented them. It's just that, if we are to believe Laurent Bossavit, that's a major undertaking. For now, then, I'll choose to believe him.
I strongly recommend this book for all with an interest in software development.
In The Leprechauns of Software Engineering, Laurent Bossavit chases down the sources of these accepted 'facts', and in the process demonstrates that there's no science behind them.
While he explains, in detail, how he manages to unravel the various citations in order to get to the heart of the matter, the book is easy to read. At times, it's like reading a thriller; I was literally turning pages, and didn't want to put the book down.
There's a risk that I suffer from confirmation bias, but this could be one of the decade's most important books about software development.
My only problem with the book is that I feel that I should not take citations for granted. Instead, in its spirit, I should get to the original sources, in order to verify that the author hasn't misrepresented them. It's just that, if we are to believe Laurent Bossavit, that's a major undertaking. For now, then, I'll choose to believe him.
I strongly recommend this book for all with an interest in software development.
December 29, 2014
This is a confusing, and ultimately quite frustrating book. The basic premise is good: the software development field is full of received wisdom that has mutated over time from simple reasonable hypotheses to being treated as established fact supported by research. But when you dig under the surface a little more, you discover that the research didn’t actually say what everyone thinks, but it’s been quoted enough times (often by people who should really know better) that everyone now assumes it must be true. So Bossavit sets out not only to debunk some of these, but (both by example, and by explicit [and somewhat patronising] training), to show you how to do likewise — or at least persuade you that you should become much more skeptical of such claims.
This is an area I know something about, having spent some time a decade or so ago digging through the original research to support the “some programmers are an order of magnitude more efficient” claims. It’s hardly a spoiler to reveal that I, like Bossavit, discovered that the original studies everyone refers to (a) were about debugging, rather than creating the original code; (b) were explicitly measuring the difference between working at a computer vs working on paper, rather than comparing like with like; (c) had a mix of programmers who were new to the language being used with experts in it; and even then, (d) had significantly less variation in results than is usually claimed; and (e) showed that the best programmer on one task could just as easily be one of the worst on some other task — hardly a ringing endorsement for the ‘super-programmer’ mythology that arose as a result.
Unpicking the threads behind myths like this can be both fun and instructive, and Bossavit does a pretty good job of it. But unfortunately the book didn’t really work for me. Partially this is because it seems still far from finished. It’s a LeanPub book, which means this is, to a certain extent, expected, and it’s explicitly noted at the start that it’s still a work in progress (and may indeed never be finished). Some chapters are barely more than a placeholder thought, still needing to be fleshed out, whereas others are disproportionately long (largely reflecting simply how much work was involved in debunking it, rather than how important or wrong it was); and there are lots of little presentational issues still needing cleaned up. The structural flow doesn’t quite work — there are too many “as we’ll see later” forward-references (and at least one spurious “as we’ve previously seen” back-reference), and the book still largely feels like a series of stand-alone articles that hasn’t yet been edited into a coherent whole.
More troubling, however, is that Bossavit can’t quite decide which position to attack in many cases: the simple “research shows …” version, or the stronger underlying claim. Even if the research doesn’t show that some programmers are twenty-eight times more productive, everyone still knows that some variation of this is true ... right? Well, er, maybe. As a throwaway at the end of the chapter on this metric, the book leaps, within a couple paragraphs, from discussing the claim, to raising the question of whether the the construct even has any validity, to outright asserting:
Similarly, when it comes to examining some of the research around bug-finding, the somewhat astonishing claim is made that:
Perhaps it is true that the entire concept of productivity of software developers is flawed, and that (outside Hopper’s moth) bugs as we think of them don’t really exist. But these are much stronger claims that go far beyond simply showing the flaws in some research, and their inclusion (with little explanation) is a little jarring. Saying “the research doesn’t say this, and even if it did it would be wrong, as the whole concept is erroneous” comes across a little too much like “My client has never even owned a gun. And even if he did, he certainly didn’t have it with him that night. And even if he did, he wasn’t the one who fired it in the bar. And anyway, he wasn’t even there.”
This is an area I know something about, having spent some time a decade or so ago digging through the original research to support the “some programmers are an order of magnitude more efficient” claims. It’s hardly a spoiler to reveal that I, like Bossavit, discovered that the original studies everyone refers to (a) were about debugging, rather than creating the original code; (b) were explicitly measuring the difference between working at a computer vs working on paper, rather than comparing like with like; (c) had a mix of programmers who were new to the language being used with experts in it; and even then, (d) had significantly less variation in results than is usually claimed; and (e) showed that the best programmer on one task could just as easily be one of the worst on some other task — hardly a ringing endorsement for the ‘super-programmer’ mythology that arose as a result.
Unpicking the threads behind myths like this can be both fun and instructive, and Bossavit does a pretty good job of it. But unfortunately the book didn’t really work for me. Partially this is because it seems still far from finished. It’s a LeanPub book, which means this is, to a certain extent, expected, and it’s explicitly noted at the start that it’s still a work in progress (and may indeed never be finished). Some chapters are barely more than a placeholder thought, still needing to be fleshed out, whereas others are disproportionately long (largely reflecting simply how much work was involved in debunking it, rather than how important or wrong it was); and there are lots of little presentational issues still needing cleaned up. The structural flow doesn’t quite work — there are too many “as we’ll see later” forward-references (and at least one spurious “as we’ve previously seen” back-reference), and the book still largely feels like a series of stand-alone articles that hasn’t yet been edited into a coherent whole.
More troubling, however, is that Bossavit can’t quite decide which position to attack in many cases: the simple “research shows …” version, or the stronger underlying claim. Even if the research doesn’t show that some programmers are twenty-eight times more productive, everyone still knows that some variation of this is true ... right? Well, er, maybe. As a throwaway at the end of the chapter on this metric, the book leaps, within a couple paragraphs, from discussing the claim, to raising the question of whether the the construct even has any validity, to outright asserting:
The 10x claim is “not even wrong”, and the question of “how variable is individual programmer productivity” should be dissolved rather than answered.
Similarly, when it comes to examining some of the research around bug-finding, the somewhat astonishing claim is made that:
while germs are very much in the “don’t go away if you stop believing in them” category, that’s much less true of these things we call “bugs”
Perhaps it is true that the entire concept of productivity of software developers is flawed, and that (outside Hopper’s moth) bugs as we think of them don’t really exist. But these are much stronger claims that go far beyond simply showing the flaws in some research, and their inclusion (with little explanation) is a little jarring. Saying “the research doesn’t say this, and even if it did it would be wrong, as the whole concept is erroneous” comes across a little too much like “My client has never even owned a gun. And even if he did, he certainly didn’t have it with him that night. And even if he did, he wasn’t the one who fired it in the bar. And anyway, he wasn’t even there.”
July 15, 2019
This book debunks many myths in software engineers, it's a great read and challenges what we know about our industry. I recommended this to a friend of back and he said: "Nobody should write anything about software engineering without reading this book."
I read this book in 2013 and really liked it; I can't talk about the updates since 2013.
I read this book in 2013 and really liked it; I can't talk about the updates since 2013.
June 7, 2019
This is one of those books that should have been a blog post (or few).
The author argues some “established facts” in software engineering and the papers that spawned them. I’ll list them below:
- The cone of uncertainty
- Uber programmers are 10 times more productive than simpletons
- Waterfall doesn’t suck (it was invented to have something to compare with newer and better development life cycles)
- The later you find a bug, more costly it is to fix
That’s all. I don’t know if you have any strong feelings about these “facts” but apparently they are mostly based on some expert’s gut feeling or problematic experiments.
The book mainly talks about the work the author had to do to uncover the truth. It gets a bit predictable after the first time so I decided to skim a bit. I was hoping for some counter examples later in the book so I could learn about what we can really measure scientifically in software development. Unfortunately I found myself skimming to the end of the book and to make it worse: it ended with a cliffhanger.
My disappointment was immeasurable, and my day was ruined.
Personally I’m guessing a sequel 10 years from now when we find out the Agile experts (the community the author belongs to) were selling us snake oil all the time and we totally ate it up. Talk about foreshadowing!
The author argues some “established facts” in software engineering and the papers that spawned them. I’ll list them below:
- The cone of uncertainty
- Uber programmers are 10 times more productive than simpletons
- Waterfall doesn’t suck (it was invented to have something to compare with newer and better development life cycles)
- The later you find a bug, more costly it is to fix
That’s all. I don’t know if you have any strong feelings about these “facts” but apparently they are mostly based on some expert’s gut feeling or problematic experiments.
The book mainly talks about the work the author had to do to uncover the truth. It gets a bit predictable after the first time so I decided to skim a bit. I was hoping for some counter examples later in the book so I could learn about what we can really measure scientifically in software development. Unfortunately I found myself skimming to the end of the book and to make it worse: it ended with a cliffhanger.
My disappointment was immeasurable, and my day was ruined.
Personally I’m guessing a sequel 10 years from now when we find out the Agile experts (the community the author belongs to) were selling us snake oil all the time and we totally ate it up. Talk about foreshadowing!
June 21, 2016
Are you in IT? If so, you must read this book. There're a plenty of fallacies circulating which are based on unverified statements. This book busts most popular, but which is more important it teaches an industry newcomer or even a seasoned professional to doubt bold sayings and ask right questions to figure out what forms the foundation of those sayings.
February 7, 2015
Bossavit does a creditable job of shining a bright light on often-repeated "foundational truths" of the software engineering field. In so doing, he uncovers troubling inconsistencies in the ways these "truths" were arrived at, documented, illustrated, cited, interpreted, and applied in software industry research publications. As previous reviewers have noted, the book's content is rather disorganized and incomplete, and at times its author seems hesitant about what to make of his own findings. Even so, I applaud this effort at beginning a worthwhile conversation about what is known of the work of building software -- and what has too long passed for well-researched "findings" in this rather poorly understood field. Should be read by anyone who works in the industry.
August 3, 2015
Well researched debunking of some of the most stubborn myths in software engineering, combined with a thorough case for critical thinking in software engineering. For some reason, I was constantly reminded of Facts and Fallacies of Software Engineering as one of the possible offenders. I should probably reread that.
If I have to pick a negative point with this book, it's the quality of writing, which is easily overcome by the quality of the content.
If I have to pick a negative point with this book, it's the quality of writing, which is easily overcome by the quality of the content.
January 7, 2016
This is a must-read. It shows how the Waterfall, the 10x developer, the cone of uncertainty and other elements of software development are not based on engineering or evidence but ultimately folklore that somehow build up. It is very interesting to follow the process how fiction became fact. I can highly recommend it! It help me a lot to question what we are doing in our industry.
The only thing missing for 5 stars is that the book is not completed yet and according to the author might never be. I think that is unfortunate.
The only thing missing for 5 stars is that the book is not completed yet and according to the author might never be. I think that is unfortunate.
February 2, 2020
pretty much what I got from this is that you can't trust a lot of meme rules of thumb in software engineering. the science sucks and is not a good fit for our industry. our work is too complex and hard to measure. the examples give some ideas about how you can debunk a lot of software engineering research and call out the BS. it's a short book, dense and not an easy read and a little expensive but worth it for originality. it pairs well with software xrays which shows a better alternative to how we can measure things in our own software developments.
August 23, 2013
A brilliant book. While the presentation could use some touching up, the contents are well worth the modest Leanpub price.
This should be mandatory reading to anyone who has even a passing interest in software development (methods).
This should be mandatory reading to anyone who has even a passing interest in software development (methods).
September 28, 2019
TL;DR - Bassavit demonstrates that several widely-cited claims about software engineering are not really rooted in convincing empirical evidence.
I'm impressed by the research and the clarity of thought. He quite successfully takes down these "leprechauns", I think, but (as he admits) he doesn't construct any sort of alternative epistemology. He weakly suggests that we should raise our standards for empirical research, and also maybe borrow techniques from the social sciences.
I think Bassavit is a little bit too dismissive of "anecdote" and "folk wisdom" possessed by professional programmers. He is right in condemning those who present e.g. "the cost of defects is higher the later they are discovered" as having the authority of empirical science behind it -- but that doesn't necessarily mean there is no other basis for software engineers to subscribe to this idea.
Economists, for example, are not afraid to posit generalizations about how the economy works based purely on mathematical models they think plausibly capture characteristics of real markets, in situations where the empirical evidence is lacking or ambiguous, or when the mechanism they are analyzing is suspected of being too non-linear/complicated for a statistical test to be plausible.
In fact, they will often present their arguments using "demand curves" and "production possibility frontiers" generated from theory, not from data, that resemble in some ways the "cone of confidence" and upward sloping "cost of defect discovery" diagrams that Bassavit is so affronted by.
The lesson I would take away from Bassavit's deconstruction is:
- stop attributing any inherent authority to scholarly citations
- stop pattern-matching on "the scientific method" as a substitute for critical reasoning
- devote less effort to the dubious project of imitating the methods of experimental science, and more effort to improving our mental models of software engineering by introspective, not empirical means.
I'm impressed by the research and the clarity of thought. He quite successfully takes down these "leprechauns", I think, but (as he admits) he doesn't construct any sort of alternative epistemology. He weakly suggests that we should raise our standards for empirical research, and also maybe borrow techniques from the social sciences.
I think Bassavit is a little bit too dismissive of "anecdote" and "folk wisdom" possessed by professional programmers. He is right in condemning those who present e.g. "the cost of defects is higher the later they are discovered" as having the authority of empirical science behind it -- but that doesn't necessarily mean there is no other basis for software engineers to subscribe to this idea.
Economists, for example, are not afraid to posit generalizations about how the economy works based purely on mathematical models they think plausibly capture characteristics of real markets, in situations where the empirical evidence is lacking or ambiguous, or when the mechanism they are analyzing is suspected of being too non-linear/complicated for a statistical test to be plausible.
In fact, they will often present their arguments using "demand curves" and "production possibility frontiers" generated from theory, not from data, that resemble in some ways the "cone of confidence" and upward sloping "cost of defect discovery" diagrams that Bassavit is so affronted by.
The lesson I would take away from Bassavit's deconstruction is:
- stop attributing any inherent authority to scholarly citations
- stop pattern-matching on "the scientific method" as a substitute for critical reasoning
- devote less effort to the dubious project of imitating the methods of experimental science, and more effort to improving our mental models of software engineering by introspective, not empirical means.
November 20, 2020
With a provoking name, "leprechauns", the author claims that we assume a lot of non-validated "truths" and calls for a critical approach. These leprechauns come in different forms and in the book there are some clues check references and validate them.
The ones explored in the book are the cone of uncertainty, the impact of TDD, 10x developers, the invention of the waterfall "method", the software crisis and cost of defects.
Question everything, if you were not doing that already.
It's a very interesting read with pointers to other books.
I didn't like that the book declares itself as "work in progress", even though the author did his last update on 2017. How could we reference it, if it may change?
The ones explored in the book are the cone of uncertainty, the impact of TDD, 10x developers, the invention of the waterfall "method", the software crisis and cost of defects.
Question everything, if you were not doing that already.
It's a very interesting read with pointers to other books.
I didn't like that the book declares itself as "work in progress", even though the author did his last update on 2017. How could we reference it, if it may change?
August 15, 2022
I liked the book - it’s pretty easy to read (if you don’t check the detailed research and follow all the references and cited sources), and is tackling an important problem we have in all the software related fields - there are a lot of pseudoscientific theories and assumptions thrown around, that we got used to take for granted. So it’s extremely helpful when someone is raising a red flag and makes you stop and reevaluate your biases and assumptions.
What I feel is missing and I would love to see (at least in a future iteration), is the mention of some valid data and “good” studies. Otherwise you leave with the feeling that everything is a sham in this field and no real studies were made in all this time with the exception of all the bad ones mentioned in the book.
What I feel is missing and I would love to see (at least in a future iteration), is the mention of some valid data and “good” studies. Otherwise you leave with the feeling that everything is a sham in this field and no real studies were made in all this time with the exception of all the bad ones mentioned in the book.
October 10, 2019
If you've ever felt like there is something fundamentally wrong with the way software is developed in a project/enterprise environment, then this book is for you. The author shows you how a few cornerstones of the software engineerings discipline are fundamentally flawed. It's a pretty quick read and I enjoyed it.
April 28, 2020
I wouldn't say this was a great book. However, it deserves credit as being one of the only books I've come across to head-on attack various empirically dubious claims found in software engineering and project management. For this, it deserves credit.
Disclaimer: author is some woke crazy dude.
Disclaimer: author is some woke crazy dude.
November 9, 2022
Enlightening and easy to read. Although it's a short book, the message is very clear: software engineering publications are plagued with myths that have been perpetuated over the years.
Don't quote me on this :)
Don't quote me on this :)
December 22, 2023
More of an extended blog post than a book. Mostly the level of research and fact checking of software beliefs that a teenager could have written, but it's still nice for someone to have done the research and written it up.
February 5, 2025
This was an entertaining read and really helps to re-think some popular opinions like the benefit of test-driven and especially the cost curve. Still, more often it is difficult to find the legit sources to an article. [citation needed]
February 23, 2020
The first half is interesting. Afterwards it starts hard to follow. But still few interesting points to think about.
March 12, 2020
Great content! My only critique was that it was too short. More leprechauns!
January 9, 2021
It kills some myths, and it encourages critical thinking.
January 30, 2022
This book gave me a different point of view about software development. The idea that software development has social and cognitive parts made me think.
April 10, 2018
Leprechauns are mythical figures that don’t exist. When you work it the IT sector, you will hear many stories about how the industry works and many people will call them engineers. That all sound so scientific and fact based that you can easily overlook one important fact: Most facts are just fairy tales. There are studies over studies, but is it true when people often enough repeat a myth?
Laurent Bossavit spend time to figure out the facts behind the myths. He not only gives you the ugly truth about the beloved ideas of the 10x programmers, the cone of uncertainty and the origin of the waterfall process, he explains how those myths are fabricated and gives you a guide to debunk other myths.
Be warned, many ideas you believe in don’t have the solid scientific ground everyone says. But if you want to be a software engineer, then it’s time to face reality and start digging for the data. Too often critical parts are missing and only through people like Laurent we can fix this situation. This book helps and is a must-read for everyone claiming to be a software engineer.
Laurent Bossavit spend time to figure out the facts behind the myths. He not only gives you the ugly truth about the beloved ideas of the 10x programmers, the cone of uncertainty and the origin of the waterfall process, he explains how those myths are fabricated and gives you a guide to debunk other myths.
Be warned, many ideas you believe in don’t have the solid scientific ground everyone says. But if you want to be a software engineer, then it’s time to face reality and start digging for the data. Too often critical parts are missing and only through people like Laurent we can fix this situation. This book helps and is a must-read for everyone claiming to be a software engineer.
July 10, 2013
Book shows how claims such as 10x productivity, cone of uncertainity, cost of change curve developed overtime and how they are a result of a game of telephone between the past and the present. Based on this book it does seem like the software industry has an issue with properly sourcing and interpreting previous studies.
Of course, if I just took this book at face value I wouldn't really be following the book's own advice of investigating claims. Unlike the papers he references in the book he actually sources well so it probably wouldn't be too hard to follow up and confirm what he claims in this book.
Pretty short read. Will probably make you disgusted with how crappy our "scientific" studies are and how people use the results.
Of course, if I just took this book at face value I wouldn't really be following the book's own advice of investigating claims. Unlike the papers he references in the book he actually sources well so it probably wouldn't be too hard to follow up and confirm what he claims in this book.
Pretty short read. Will probably make you disgusted with how crappy our "scientific" studies are and how people use the results.
January 14, 2016
An excellent read which sums up a lot of thoughts I've had about software engineering as a profession.
It tackles a lot of "common knowledge" and how their origins suggest that much of our "common knowledge" is built on a weak foundation of studies. For example, the idea of the 10x programmer, or the cost of defects based upon which they are detected in a project are both tackled as having come into the common knowledge on a flawed background.
The book itself has a bit of an odd structure, but the content is good enough to make up for this. I suggest it to all software engineers.
It tackles a lot of "common knowledge" and how their origins suggest that much of our "common knowledge" is built on a weak foundation of studies. For example, the idea of the 10x programmer, or the cost of defects based upon which they are detected in a project are both tackled as having come into the common knowledge on a flawed background.
The book itself has a bit of an odd structure, but the content is good enough to make up for this. I suggest it to all software engineers.
September 22, 2013
<3 if you want to shatter some common truths of software engineering, this is good book to start. Also good example of how to disect/analyze "studies say" given facts. Bit short, but book is still evolving.
August 29, 2021
Important read for anyone interested in software engineering as a profession.
October 13, 2016
Some good examples of analyses for understanding what's fact and what's a leprechaun.
April 4, 2017
It’s an inspiring read, but I’m not convinced the whole idea is worth a book. Basically the author shows how flawed the “software engineering” research is and how to detect bogus claims and verify sources – this could well fit into a short series of blog posts. There are just a few analysed examples of claims not backed by proper evidence – I was expecting a more structured analysis with some summary. I found the book one of the hardest to read when it comes to the language itself – and it wasn’t necessary, because the subject is not hard to explain. I suppose either the author really wanted to sound smart, so tried hard to use sophisticated and unpopular words, or lacked the proficiency in English enough to write simple things in simple language. I wish I read something shorter in this topic instead.
Read
December 16, 2017An interesting book but the topic of Software Engineering is more of a context - the book is more focused on how research is miss-used and that we should consider it like "Chinese whispers" and check with the original source before we make assumptions on it.
Displaying 1 - 30 of 31 reviews