Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making

by Gary Klein

An expert explains how the conventional wisdom about decision making can get us into trouble—and why experience can't be replaced by rules, procedures, or analytical methods. In making decisions, when should we go with our gut and when should we try to analyze every option? When should we use our intuition and when should we rely on logic and statistics? Most of us would probably agree that for important decisions, we should follow certain guidelines—gather as much information as possible, compare the options, pin down the goals before getting started. But in practice we make some of our best decisions by adapting to circumstances rather than blindly following procedures. In Streetlights and Shadows , Gary Klein debunks the conventional wisdom about how to make decisions. He takes ten commonly accepted claims about decision making and shows that they are better suited for the laboratory than for life. The standard advice works well when everything is clear, but the tough decisions involve shadowy conditions of complexity and ambiguity. Gathering masses of information, for example, works if the information is accurate and complete—but that doesn't often happen in the real world. (Think about the careful risk calculations that led to the downfall of the Wall Street investment houses.) Klein offers more realistic ideas about how to make decisions in real-life settings. He provides many examples—ranging from airline pilots and weather forecasters to sports announcers and Captain Jack Aubrey in Patrick O'Brian's Master and Commander novels—to make his point. All these decision makers saw things that others didn't. They used their expertise to pick up cues and to discern patterns and trends. We can make better decisions, Klein tells us, if we are prepared for complexity and ambiguity and if we will stop expecting the data to tell us everything.

Genres: Psychology, Nonfiction, Business, Science, Self Help, Leadership, Management

355 pages, Kindle Edition

First published September 4, 2009

About the author

Gary Klein, Ph.D., is known for the cognitive models, such as the Recognition-Primed Decision (RPD) model, the Data/Frame model of sensemaking, the Management By Discovery model of planning in complex settings, and the Triple Path model of insight, the methods he developed, including techniques for Cognitive Task Analysis, the PreMortem method of risk assessment, and the ShadowBox training approach, and the movement he helped to found in 1989 — Naturalistic Decision Making. The company he started in 1978, Klein Associates, grew to 37 employees by the time he sold it in 2005. He formed his new company, ShadowBox LLC, in 2014 and is the author of five books.

The Lightbulb Moment: Insights are unexpected shifts in the way we understand how something works, and how to make it work better. The talk examines two mysteries. First, where do insights come from? This talk presents a new account of the nature of insights. Second, how can we trigger more insights? This talk describes a strategy for adopting an insight mindset.

Reviews

[Dorum · Rating 5 out of 5]

Why isn't this as famous and well known as Thinking Fast and Slow? Or... Antifragile? To be honest he is not as quotable as Taleb. Neither is he as simple to read as Kahnemann. But he is every bit as relevant.

[Raz Pirata · Rating 4 out of 5]

“Our ability to obtain information has overwhelmed our ability to make sense of it”

For me, Gary Klein is the Johnny Ramone is Decision Theory and although that comparison may horrify him I offer it with the greatest amount of respect. He gives it to us straight. No fluff or flights of fancy, just the goods.

In Streetlights and Shadows, Klein argues against 10 (common theoretical) Claims About How to Think More Effectively. He presents the argument of each claim and then goes to work punching holes in the claims by providing a relentless barrage of anecdotes and evidence that you could drive a tour bus through.

“We have to stop treating decisions as gambles... this view assumes that the decision maker is passively awaiting the outcome of the gamble rather than actively managing the situation and shaping the options”

This is not just a seek and destroy mission, however. What Klein does is offer a reconstruction amongst the rubble. Replacing each claim with something seemingly intuitively obvious and I believe that is the whole point.

Streetlights and Shadows continues along his journey to show appreciation for how we actually decide in the real world of our everyday lives and how those processes are effective. This book is a continuation of Klein’s regard for intuition, which he began in Seeing What Others Don’t, and expertise, as evidenced in Sources of Power.

“Reason has constructed two aspects of human performance...in treatment of errors: the human as a hazard and the human as a hero”

With a utilitarian literary structure and an abundance of examples and evidence, Streetlights and Shadows offers both a deep and wide examination of how we think best. It provides many insights into the things we do well and where some current decision theory falls short. It explains how our biases and reliance on heuristics do not make us irrational but are tools to formulating reasoned and adaptive approaches to solving problems and reaching our goals.

“We shouldn’t trust either analysis or intuition. When they give us different answers, we should use this as a signal to re-examine each one”

Streetlights and Shadows makes no apologies for its place in the world of Decision Theory and Strategic Analysis. It takes no sides with either proponents of the Naturalistic or Analytical models. It presents a clear picture into the complex and ambiguous domains which require us to think and decide and offers a ray of light. As straight forward and in your face as Johnny's down strums, this book cuts to the core. Gary Klein is a thinker's punk rocker.

Overall Score: 4.4 / 5.0

In a sentence: The good and bad about how we actually make decisions

[Scott Ford · Rating 5 out of 5]

Wouldn't it be great if we could make decisions within the same vacuum that many advice gurus use to establish their theories! Unfortunately, mere mortals have to contend with ambiguity, contradiction and redundancy on a regular basis. Gary Klein places the whole process of decision making into context. A great book.

[Christopher Widjaya · Rating 4 out of 5]

good book regarding human thinking on decisions. but too draggy at parts with countless rereferences to anecdotes, sort of beating around the bush for a good bit, but never fails to put the points down in the ground firmly. a good read, but a hundred pages less is possible and maybe more is less is really the best.

[Matt · Rating 3 out of 5]

While the author does a decent job of debunking carefully crafted statements that seem meant to be debunked, I’m not convinced any of his conclusions are as revolutionary as they seem to be in the book. Most of his evidence comes from a few examples rather than a detailed analysis of a broad variety of situations. Ultimately, he seems to take a statement that most people would say, “yeah, that’s usually true” and points at a case or two to prove that the statement isn’t always true. Well, it seems obvious to me that most people didn’t claim those statements are always true, that there are always exceptions. That said, after wading through the fodder of examples and the poorly constructed arguments, there is truth in his conclusions and anyone can stand to strengthen or correct their mental models by reading this book. Would recommend with reservation, but will never reread.

[Hans De Leenheer · Rating 5 out of 5]

Probably the best book I've read so far in the intricacies of decision making. How do we think we make decisions and how do we actually make decisions? How do decision-making principles become myths that do not apply when faced with more complex domains?

[Jill · Rating 4 out of 5]

In Streetlights and Shadows, Klein argues that the traditional claims for thinking and making decisions only apply in well-ordered situations (under "streetlights"). In situations that are ambiguous, complex and unpredictable (in the "shadows"), however, these processes are at best useless and at worst, work against us. He identifies what he considers to be the ten most worrying of the common pieces of advice given by researchers, organisational developers and management specialists on decision-making, then systematically unpacks why these claims are ineffective and what we might do instead.

I'd just read Steven Johnson's Farsighted: How We Make Decisions That Matter the Most before picking up Streetlights and Shadows (they were on the same shelf in the library). Klein's book, apart from being more tightly and convincingly written, pretty much tells you that Johnson's advice, while not unhelpful, has its limits. In a complex situation, Klein would argue that things like full spectrum analysis and influence diagrams can be helpful but we shouldn't hold on to them too tightly.

Claim #1: Teaching people procedures helps them perform tasks more skilfully.
Klein argues that procedures are only useful in well-ordered situations when they can substitute for skill, not augment it. Procedures are also useful as training tools to help get novices started in learning a task, as memory aids, or to impose consistency for teams, particularly ad hoc teams. In complex situations, however, procedures are not a substitute for experience. For these, Klein suggests that "people need judgement skills to follow procedures effectively and go beyond them when necessary", e.g. senior pilots overriding standard procedures when they see fit. Indeed, emphasising procedures over skill might lead to a situation where you get consistently mediocre performance, as there is no incentive for people to try something different and potentially better.

Claim #2: Decision biases distort our thinking.
Klein notes that people use heuristics that are generally effective but aren't perfect. His sense is that these biases are not as problematic in real world settings, compared to experimental findings. Rather than discouraging people from using heuristics, we should "help them build expertise so they can use their heuristics more effectively. Putting judgements into perspective, such as by using the premortem technique, and representing data in ways that support intuition (e.g. using frequency data rather than probabilities), can also help.

Claim #2a: Successful decision makers rely on logic and statistics instead of intuition.
In a similar vein, Klein argues that it's not an either/or situation where we should only use logic and statistics or intuition. Rather, we need to "blend systematic analysis and intuition".

Claim #3: To make a decision, generate several options and compare them to pick the best one.
Klein argues that in reality, we rarely use formal methods of decision making, comparing and weighing different options. It requires too much time and effort. Rather, good decision makers use their experience to recognize effective options and evaluate them through mental simulation.

Claim #4: We can reduce uncertainty by gathering more information. Too much information can get in our way.
Klein makes a distinction here between puzzles and mysteries. While gathering more data can help us to solve puzzles, mysteries require more sense making than data. In complex situations, more data actually increases uncertainty, as the marginal value of additional data points falls while increasing complexity. In cases like Pearl Harbor, 9-11 and Enron, decision makers ignored weak signals and instead explained them away when these didn't fit in with their mental models. "The data was there but the sense making broke down." Instead of devoting resources and energy to collecting more information, we should pay attention to how we sort that information, share and integrate it for sense making.

Claim #5: It's bad to jump to conclusions - wait to see all the evidence.
Klein argues that keeping an open mind makes us passive, by waiting to see what other options and information might surface, rather than engage in anticipatory thinking. . This claim makes us "slaves to the flow of information" and together with Claim #4, can paralyse the decision maker. Instead, Klein argues, we should speculate and test our speculations instead of committing to them.

Claim #6: To get people to learn, given them feedback on the consequences of their actions.
Klein says that feedback is difficult to understand in complex situations and it doesn't help if the learner doesn't understand its implications. What happens if we're not getting feedback that is helpful to us, for instance receiving outcome feedback instead of process feedback? Or if people don't notice and attend to the feedback? Or if it's challenging giving feedback because pertains to tacit knowledge? It is not sufficient to give feedback but we have to also find ways to make it understandable for the learner.

Claim #7: To make sense of a situation, we draw inferences from the data.
Klein points out that drawing inferences from the data is only meaningful if (a) we know what to look out for and what counts as a meaningful data point; and (b) our inferences allow us to form a coherent story to explain events. Our frames determine what counts as data and an experienced expert can draw useful inferences because he knows what to look out for to create his data set, compared to a novice.

Claim #8:The starting point for any project is a clear description of the goal.
Again, Klein points out that this claim only holds water in stable situations. In a complex situation where there are many unknowns, rather than try to have absolute clarity at the start and suffer from "goal fixation", we should redefine goals as we go along. Things like Gantt charts for complex undertakings are misleading in promising that adherence to a predetermined schedule is possible and a waste of resources.

Claim #9: Our plans will succeed more often if we identify the biggest risks and then find ways to eliminate them.
Klein's research indicates that executives don't bother to conduct formal risk analyses but instead try to envision specific scenarios and whether the worst possible outcomes were tolerable. Klein advocates that instead of formal risk analyses, we rely on resilience engineering - organising to anticipate, learn and adapt.

Claim #10: Leaders can create common ground by assigning roles and setting ground rules in advance.
Klein points out that while you can create common ground at the start, this common ground rapidly erodes as the situation changes. Common ground therefore has to be repeatedly re-established and all team members should be responsible for continually monitoring the common ground for breakdowns and repairing it when necessary.

In his book, Klein draws on a wide range of research and cases - from firefighters to weather forecasters, healthcare professionals to pilots and military personnel - to illustrate his points. A fascinating and thought provoking read.

[Marco · Rating 5 out of 5]

A great book on decision making in conditions of uncertainty and ambiguity, rooted in a naturalistic approach. While giving a good overview of the territory, Klein adds his unique voice and approach, explained in simple ways, and gives concrete advice while busting some myths of a certain rationalistic school of decision making. Recommended

[Chrcgr · Rating 2 out of 5]

Some thought provoking content, but could easily have been condensed to about a dozen pages.

[João Panizzutti · Rating 3 out of 5]

I'm going to be honest. I didn't finish this book. Stopped at chapter 7 because Gary uses way too many slow, boring stories to make his points. But here's the thing: the ideas are so good that I still took extensive notes and I'm still recommending it.

This is basically the practical companion to Thinking Fast and Slow. Where Kahneman tells you how thinking works, Klein tells you how experts actually make decisions in the real world. It's dense, it's slow, but if you can push through Gary's storytelling style, the framework is worth it.

Gary is an amazing researcher but no the best writer.

Here are a few of my notes

The book challenges a widely accepted myth: that algorithms and formulas always outperform expert judgment. Turns out, that's bullshit. The studies showed both experts AND algorithms sucked at prediction. The real insight? Predicting complex domains is just really hard. But that doesn't mean expertise is useless.

Here's what Klein argues: humans are excellent at finding the right data using intuition, but we're terrible at aggregating it into conclusions. Algorithms are good at aggregating data but can't find the unseen variables that matter. The solution isn't choosing one over the other. It's using both.

The Recognition, Primed Decision Model

This is how actual experts make decisions in the real world. The better the expert, the fewer options they consider. Beginners evaluate 10 different choices. Experts recognize the pattern, pick the first good solution, and move. They run a mental simulation, and if it works, they execute.

This connects directly with what [Nassim Taleb] talks about. The dangerous is in the unseen. The 2008 crisis happened because algorithms failed to account for variables they couldn't measure. LTCM, the hedge fund run by mathematicians, went bust the same way. Their models were perfect until they hit a scenario with ruin risk.

Why Procedures Fail Experts

The book has an entire chapter on this, and it's crucial. Procedures raise beginners to an intermediate level. That's good. But they destroy expert performance because they ignore tacit knowledge, the stuff experts know but can't write down.

A chess master knows exactly when to break the "rules." A beginner following the same rules will use them everywhere, even when they don't apply. Same in the gym: beginners need strict form cues, but experts know when to cheat reps, when to recover, when to push volume.

The problem is when procedures become mandatory. You get [The Overfitting Mental Model], systems too rigid to adapt to novel situations. Companies that force procedures on everyone end up keeping beginners and losing experts.

Conscious vs Unconscious Decision-Making

Here's where it gets interesting. For simple choices, rational deliberation works. But for complex domains, thinking too much makes decisions worse. Your brain is incredibly good at pattern recognition and detecting signals you can't articulate. When you force everything into conscious analysis, you miss what's hard to put into words, often the most important stuff.

This is what [Naval Ravikant] means when he says to make decisions with your gut. Your brain processes data subconsciously and gives you a signal. Ignoring that because it's not "rational" is stupid.

The Right Approach

Klein's conclusion: blend intuition and analysis. Use your expertise to gather the right data and recognize patterns. Use algorithms to aggregate and test your intuition. Update your priors with new information, like in [Bayesian Statistics].

Don't use analysis as a one-size-fits-all solution. Use it to improve your thinking, not replace it.

Why This Matters

The book connects to everything I've been thinking about lately. It's the same framework as [Skin in the Game]. Rationality is surviving, and if a heuristic kept organisms alive, it's not irrational. Experience in uncertain, high-stakes environments creates real expertise.

Most decision-making advice comes from researchers studying bounded problems with clean data.

That's not the real world. In the real world, you don't have all the variables. You're operating in the shadows, not under streetlights. That's when expertise matters most.

If you work in any domain where you need to make decisions under uncertainty, read this.

[ROHITH R NAIR · Rating 5 out of 5]

TLDR: If you liked Thinking,Fast and Slow and are looking for another book that is just as insightful,something that can enrich your mental models,this is the book you've got to read.


As a child,I always wondered,what is the most efficient way of performing a given task and due to the abundance of procedures and pre made steps that one is exposed to,even during the high school days there's always an importance placed on the widely accepted 'to do' methods.

As a teenager,it dawned upon me that these 'Guidelines' for performing actions worked pretty well(In my defense,most of the academic situations I was exposed to at that age were the 'Simple' tasks and not the complex ones),however a part of me noticed that these procedures and plans didn't always go as per the planner's requirement but I didn't care about that minute subset of tasks.

As a young adult,my first exposure to the 'Shadows' were during my time in medical school,that was when I realised that I was merely living under the illusion,that the world was mostly comprised of 'Streetlights'. Ever since that time,there was always an unanswered question in my mind,how does one deal with complex situations?

I must say,this book does justice in answering just that question and I can confidently say that my mental models have undergone a significant change after reading this book,I finally understand the nuance that exists in the real world situations and the different ways we ought to deal with them.

I'm extremely grateful to Gary Klein for writing such a wonderful and insightful book,it is a must read for anyone who is on a quest to understand the world better.

[Mr Abdi Ahmed Samater · Rating 5 out of 5]

Overall excellent book with loads of insights which can crossover into many areas you may wish to work on in your life. From heuristics, the types that exists, the benefits, to popular claims regarding decision making and why it may not be as accurate as it may seem. The topic on feedback was a game changer for me (as I coach football). The feedback I now give (which is rarely given unless needed), empowers my students to learn to recognize cues for themselves, interpret situations, ask questions, formulate possible solutions, take action, then when stumped for a successful solution will I SUGGEST (as opposed to impose) the cause and effect relationship between their actions, and the subsequent events which take place, leaving a space open for interpretation by the player to then pursue different avenues (decisions) to resolve their problems via concepts/patterns recognition and situation matching. A revolutionary concept for me and it is for this reason I give this book a 5 star rating. Definite must for any type of mentor/coach etc...

[David Roberts · Rating 3 out of 5]

I liked the book but didn't like the approach. Here's my summary.

Step 1: Take this quiz to see if you agree with these statements that virtually every business expert agrees with.
2. News flash: everyone agrees with these basic principles.
3. You know, they really aren't actually true.
4. Here's an example.  You all think feedback is important to improve performance. However there was this one guy who tried to land a plane on an aircraft carrier and they kept waving him off (feedback) but he didn't know why they waved him off and didn't figure it out until an expert came and walked him thru the process (which apparently doesn't count as feedback to the author but to me is precisely what good feedback is).
5.  So based on this one example, feedback is ineffective.
6. Same for the other 9 things that everyone believes.  All untrue.
7. Well, actually they are true, really, but you have to be careful when things get complex.

Since I already knew #7, I didn't like being dragged thru steps 2-6.  It felt disingenuous and also I thought that maybe the author just might be coming from a place where he thinks he is smarter that some of the world's greatest business minds. Like Peter Drucker on strategy. Not saying this is true, but it was an undercurrent I felt I detected.  Plus no one is smarter than PD when it comes to strategy.

Just my 2 cents and still glad I read the book. 

[Toshi · Rating 5 out of 5]

We are in complex world, not a simplistic world that was until 30 years ago. Decision making as a person, or in a project, in organization, has not been evolved to adapt to today's complex situations. Most of us agree with the statement "successful decision makers rely on logic and statistics instead intuition", "we can reduce uncertainty by gathering more information", "the starting point for any project is to get a clear description of the goal", "our plans will succeed more often if we identify the biggest risks and find ways to eliminate them." If you agree with those statement, this book is for you. Again, we are in complex world. This book teaches how we are wrong to think linear, simplistic. Each of us need to realize we have psychological inertia.

[Rajesh Iyer · Rating 5 out of 5]

Having read a lot of books on decision theory, I was seduced into believing that the best decisions (in business and work) are made from sexy statistics and graphs. Turns out it’s partial bullshit after reading this book. This book clearly explains the difference between taking decisions in stable and predictable scenarios versus complex situations where at times you don’t even know what you are looking to resolve. Your experience and the intuition you develop with it matters and not something you ignore but fall back on. People in the world of business ought to read this as I have always wondered how data dependant senior management make such poor decisions armed with all that data…Now I know why and no one does a better job of this than Gary Klein. Gotta read more of his books…

[Mihai Cosareanu · Rating 3 out of 5]

I didn't learn anything new from the book. I've read many others before that contain somehow parts of the advice offered in the book. The book is all about myths of decision making when you try to make decisions in an unordered, complex, and surprising domain.

I don't know why I found it hard to stick to the book, maybe it's the writing ... I struggled to finish it, but I find that the content is quite good if you're new to "how people make decisions".

[Dan · Rating 5 out of 5]

I have developed an interest in decision making and this book has provided me with more insight after having read Thinking Fast and Slow by Kahneman.
Klein adds some interesting thoughts that are handled differently by the Kahneman book and I will continue finding other relevant books on this topic as a result.

[Mohaimin Khan · Rating 5 out of 5]

One of the best book on the cognitive decision making process. I really like the idea how Klein has used the metaphor of streetlights and shadows to illustrate the complex dynamics of decision making process. The 10 claims on decision making, real life examples of why and how they did not work and a better cognitive based approach - made the book really easy for all kind of readers.

[William "Spig" · Rating 4 out of 5]

I really enjoyed this book. It is written in the Freak-a-Nomics style. Life and complex problems are not simple formulaic puzzles to figure out. The book shows the science behind living and leading beyond the complicated and in the complex.

[Jason Robert Bowers · Rating 5 out of 5]

This book was suggested by a friend. Not something I would normally pick up myself but I'm glad I did. I was a great read that had me thinking differently about decision making and the importance of expertise. Any one in a leadership role, or studying leadership, should give this book a read.

[Richard C · Rating 4 out of 5]

Reminded me of Enrico Fermi’s strategy of not overly complicating a problem, and solving with the most elegent factors in the decision making process. Now if someone could just quickly tell me how to recognize them I’d be in the pink!

[James Harris · Rating 5 out of 5]

6 word review: Compilation of practical decision making strategies

[Michael Kinsey · Rating 2 out of 5]

Too many anecdotes and not enough getting to the point. Sources of Power was much better and covers all the same stuff.

[Liquidlasagna · Rating 3 out of 5]

better than all those shitty pop-science books

[Jeff · Rating 4 out of 5]

Klein's discussion of ten common claims relative to decision-making in organizations is an interesting take on a frequently-covered topic. The author goes to great lengths to indicate a state of 'partial disagreement' with the claims, noting that most work in well-ordered situations yet break down in complex and/or ambiguous environments. Rather than writing a handbook on decision-making, what Klein is really doing is lending support to the notion that humans and the sciences that study them cannot be entirely reduced to formulas, checklists, and statistics. And with that, I whole-heartedly agree.

There are times when Klein is overselling his point. Chapter 16 regarding common ground is one such instance. He offers one grand insight for the chapter - and it is a good one - but then needs to describe it, re-describe it, and tell it again.

Overall, though I read this for a course on executive decision-making, I found myself genuinely enjoying it and mentally applying it to the contexts in which I find myself. What more can we really ask from our non-fiction?

[Yadira Denisse · Rating 3 out of 5]

I actually could not finish reading this book. I read 3/4 because my brain got tired of trying to interpret what it was saying. I think is a great read!!!! It basically explains everything you think you know and then tells you not to believe it and why...it makes perfect sense but it's academically written. I was trying to read it for pleasure and my brain couldn't handle it!

Nevertheless, I should also mention that as a Law Enforcement Officer most of his theories about why I should deny the initial assumptions make perfect sense. I found myself agreeing with all his as theories and understanding the reasoning behind his explanations (his better assumptions).

[Muhammad al-Khwarizmi · Rating 4 out of 5]

Very decent book though the author gets overly verbose at points. In the chapter on risk management for example it was very obvious that he was talking about so-called "Knightian" uncertainty and I wished he would just come out and say it. I would say Klein certainly mounted an illuminating challenge to conventional decision and management theory anyway. I'm loath to quote John Lennon but "life is what happens when you're making other plans". That being said, I still am not fully convinced of the degree to which he raises intuition above analysis, though both are clearly important.

[Ben Pratt · Rating 5 out of 5]

Outstanding insight into problem solving under complex conditions

Walks through the most common assumptions about what should be done to solve problems and drive continuous improvement, showing how these generally hold up well under simpler, more straightforward conditions (solving puzzles) - and it so well in more complex situations (solving mysteries). Very pragmatic and useful knowledge managers and leaders in all walks of life...

[Joan · Rating 4 out of 5]

I refer to this book all of the time in training colleagues in incident response. Especially his study of pilots, when hijacked, are not able to consult their manual. Instead, they must rely on training, instincts, and adaptive decision-making to aid in mitigating serious consequences is something everyone can appreciate and practice.

[John · Rating 5 out of 5]

Every teacher should read this book. Great examples of feedback and understand how people learn and challenges simplistic concepts of learning and teaching. But great book for people that love understanding decisions and pitfalls.