Probably Overthinking It: How to Use Data to Answer Questions, Avoid Statistical Traps, and Make Better Decisions

by Allen B. Downey

An essential guide to the ways data can improve decision making.
 
Statistics are in news reports, at the doctor’s office, and in every sort of forecast, from the stock market to the weather. Blogger, teacher, and computer scientist Allen B. Downey knows well that people have an innate ability both to understand statistics and to be fooled by them. As he makes clear in this accessible introduction to statistical thinking, the stakes are big. Simple misunderstandings have led to incorrect medical prognoses, underestimated the likelihood of large earthquakes, hindered social justice efforts, and resulted in dubious policy decisions. There are right and wrong ways to look at numbers, and Downey will help you see which are which.
 
Probably Overthinking It uses real data to delve into real examples with real consequences, drawing on cases from health campaigns, political movements, chess rankings, and more. He lays out common pitfalls—like the base rate fallacy, length-biased sampling, and Simpson’s paradox—and shines a light on what we learn when we interpret data correctly, and what goes wrong when we don’t. Using data visualizations instead of equations, he builds understanding from the basics to help you recognize errors, whether in your own thinking or in media reports. Even if you have never studied statistics—or if you have and forgot everything you learned—this book will offer new insight into the methods and measurements that help us understand the world.

Genres: Nonfiction, Mathematics, Science

256 pages, Hardcover

Published December 6, 2023

About the author

Allen Downey is a Professor Emeritus at Olin College and the author of a series of freetextbooks related to software and data science, including Think Python, Think Bayes, and Think Complexity, which are also published by O’Reilly Media. His blog, Probably Overthinking It, features articles on Bayesian probability and statistics. He holds a Ph.D. in computer science from U.C. Berkeley, and M.S. and B.S. degrees from MIT.

Reviews

[Librariann · Rating 4 out of 5]

** I received an digital advance copy from the publisher, because I am a librarian and librarians are awesome**

As a data-junkie convert later in life, I didn't have much of a foundation or vocabulary for some of the basic concepts in this - for example, I understood weighting a dataset and why it should be done, but I couldn't tell you about the inspection paradox.

I found myself taking notes from this book like I was in a college class. Even though I still wouldn't pass a test on the key terms, this book helps explain the math behind key data interpretation errors (Simpsons Paradox!) in somewhat easy to follow examples.

It's no Mary Roach, and will still appeal to a niche audience, like English major Librarians who have stumbled into a data reporting and interpretation role mid-career. It's somewhere between a dry textbook and a Khan Academy class. I didn't necessarily learn everything I need to know, but now I know more about what I want to learn.

Enjoyable for nerds and nerd adjacents

[Di Wu · Rating 4 out of 5]

First read of the year is super fun!

Before reading this book: Lionel Messi is a great soccer player
After reading this book: Lionel Messi is an outlier of a lognormal distribution

[Ethan Swan · Rating 4 out of 5]

Good but not great. I found it an easy read throughout and not dry, which is a real accomplishment for a book about math. The author's analysis comes off as clear and simple but undoubtedly took a lot of effort. That said, most of the statistical "surprises" presented in the book didn't seem that remarkable to me (admittedly a person with an existing background in stats) and the writing is fine but not remarkable.

[Bryan Fox · Rating 3 out of 5]

I'd put this into the category of "people who would most benefit from reading this book are the ones who are least likely to read it" texts. Although you could probably say the same of any empirical, statistics-driven analysis of the misinterpretations of data that lead us further and further down the "Your facts are not as valid as my feelings" path which is pretty much leading to the end of an enlightened world. The examples are clear and plentiful, but this book's popularity will be/was no doubt hampered by the fact that it's not nearly as pacey/podcast-adjacent as other texts which attempt to cover similar ground. "Make it punchier! We want our data to be SEXY!"

[Steve · Rating 4 out of 5]

I enjoyed this book; it was a fun read, with some humor. The writing is clear and conversational. There were lots of examples clearly laid out. I did find some of the stats info complex and I think that this book will be better enjoyed by people with some familiarity with statistics. Thank you to Edelweiss and University of Chicago Press for the digital review copy.

[Leandro Carísio · Rating 5 out of 5]

O livro trata de muitas falácias na área de estatística e probabilidade. Os exemplos em geral são bem interessantes e instrutivos. Alguns são bem atuais.

O leitor aproveitará mais o livro se ele já tiver algum conhecimento básico sobre o assunto. Caso não tenha, em alguns casos provavelmente ficará perdido. Mas, mesmo nesses casos, acho que é uma leitura bem válida!

[Alex]

Approachable intro to a few topics in statistics and their real world applications/consequences. I’ll remember the chapters about vaccines, homophobia, and recidivism risk.

[Neils · Rating 5 out of 5]

Very nice read and also inspiring!

[Michiel · Rating 3 out of 5]

Fun book, really enjoyed the first half. The author has an infectious enthusiasm for data