Classification in the Wild: The Science and Art of Transparent Decision Making

by Konstantinos Katsikopoulos, Özgür Şimşek, Marcus Buckmann

Rules for building formal models that use fast-and-frugal heuristics, extending the psychological study of classification to the real world of uncertainty.

This book focuses on classification--allocating objects into categories--"in the wild," in real-world situations and far from the certainty of the lab. In the wild, unlike in typical psychological experiments, the future is not knowable and uncertainty cannot be meaningfully reduced to probability. Connecting the science of heuristics with machine learning, the book shows how to create formal models using classification rules that are simple, fast, and transparent and that can be as accurate as mathematically sophisticated algorithms developed for machine learning.

Genres: Psychology, Nonfiction

200 pages, Hardcover

Published February 2, 2021

Reviews

[Wej · Rating 5 out of 5]

XGBoost is the way to win a Kaggle competition with tabular data. Now try explaining this algorithm to a medical practitioner, or even in an ML job interview for that matter. This book shows that rather than relying on complex algorithms, very high performance and high explainability can be achieved with fast and frugal heuristics. These simple decision trees split data into simple paths with only use of a given feature. This approach creates decision trees that can be easy to apply and explain. Surprisingly, their performance is not terribly bad. Even simpler approaches, like tallying, can be effective and are very easy to use. The authors suggest benchmarking more complex models to these simpler versions.

The book is written by a group of academics and it shows. The style varies between chapters, but this was hard to avoid to cover wide areas of machine learning, statistics, psychology and economics. I’m afraid that this interdisciplinarity, which I see as an advantage, might mean that the book will not find as many readers as it should. These fast and frugal heuristics are unheard of in introductory (and even advanced) ML textbooks or courses. It's a pity as they are rather simple and could serve as a decent benchmark.

[Lay Len Ching · Rating 5 out of 5]

Have spent the past 5 weeks working on research involving fast and frugal trees in the context of expected utility gambles and ergodicity economic frameworks. Blog post incoming.

Great book btw.

[John Eldreth · Rating 5 out of 5]

Reality Is Relatively Simple!

I highly recommend this book if you are tackling real world classification tasks and not academic exercises. This book demonstrates how relatively simple transparent rules or tallying approaches, not only helps one understand what is going on, but often times is more accurate or at least as accurate as more complex approaches. This is because the real world is filled with uncertainty where a lot of historical variables simply do not apply to discerning the future. This matches my Sexperience with real world time series forecasting. In the majority of cases I encountered, relatively simple forecasting models like exponential smoothing and seasonal averaging of more recent data were more accurate than complex models like ARIMA, Bayesian, and neural networks. Also when it came to time series classification, I discovered simple (i.e. shallow) classification trees using a few time series statistical features were more accurate (and understandable) than deep learning classification models (i.e. multi-layer neural networks). Initially I was taken with the idea that deep learning approaches "abstract" statistical features automatically so you don't have to determine those features yourself. After much effort in trying to determine the correct deep learning architecture and tuning parameters, the classification results on unseen time series were disappointing. Basically it turned out the models were sophisticated associative memories where smallish variations in the data were not picked up. It would only correctly identify time series that were practically indistinguishable from those in the training data. The so-called abstraction ability turned out to be optimizations for looking up training data instances. So I was left with using my experience-based intuition to derive a few sensible statistical features applied by a smallish traditional classification tree to do the real world time series classification task. I also knew what was going on and why. That's why this book is a refreshing read. So much of academic research appears to be done to boost egos and tenure tracks, not tackle real world problems where relatively simple transparent models are used to do everyday tasks.

[Aaron Michael · Rating 3 out of 5]

We wrote this book against the prevailing belief that complex methods of classification are always better and more data always lead to more accuracy. Although this belief is true in situations of certainty and complete control, where the future resembles the past, it does not necessarily hold in situations of uncertainty and lack of control. In such situations, simple rules can predict more accurately on the basis of less information and are more transparent and understandable to boot. We have used the terms in the lab and in the wild as a shorthand for these situations; many real-life situations have elements of both.
A key proposition is that simple rules do well in the wild. This is the unstable-world principle, which states that if the future differs from the past in unpredictable ways, then keep it simple.