Eric Eskin

“In the EPJ results, there were two statistically distinguishable groups of experts. The first failed to do better than random guessing, and in their longer-range forecasts even managed to lose to the chimp. The second group beat the chimp, though not by a wide margin, and they still had plenty of reason to be humble. Indeed, they only barely beat simple algorithms like “always predict no change” or “predict the recent rate of change.” Still, however modest their foresight was, they had some. So why did one group do better than the other? It wasn’t whether they had PhDs or access to classified information. Nor was it what they thought—whether they were liberals or conservatives, optimists or pessimists. The critical factor was how they thought. One group tended to organize their thinking around Big Ideas, although they didn’t agree on which Big Ideas were true or false. Some were environmental doomsters (“We’re running out of everything”); others were cornucopian boomsters (“We can find cost-effective substitutes for everything”). Some were socialists (who favored state control of the commanding heights of the economy); others were free-market fundamentalists (who wanted to minimize regulation). As ideologically diverse as they were, they were united by the fact that their thinking was so ideological. They sought to squeeze complex problems into the preferred cause-effect templates and treated what did not fit as irrelevant distractions. Allergic to wishy-washy answers, they kept pushing their analyses to the limit (and then some), using terms like “furthermore” and “moreover” while piling up reasons why they were right and others wrong. As a result, they were unusually confident and likelier to declare things “impossible” or “certain.” Committed to their conclusions, they were reluctant to change their minds even when their predictions clearly failed. They would tell us, “Just wait.” The other group consisted of more pragmatic experts who drew on many analytical tools, with the choice of tool hinging on the particular problem they faced. These experts gathered as much information from as many sources as they could. When thinking, they often shifted mental gears, sprinkling their speech with transition markers such as “however,” “but,” “although,” and “on the other hand.” They talked about possibilities and probabilities, not certainties. And while no one likes to say “I was wrong,” these experts more readily admitted it and changed their minds. Decades ago, the philosopher Isaiah Berlin wrote a much-acclaimed but rarely read essay that compared the styles of thinking of great authors through the ages. To organize his observations, he drew on a scrap of 2,500-year-old Greek poetry attributed to the warrior-poet Archilochus: “The fox knows many things but the hedgehog knows one big thing.” No one will ever know whether Archilochus was on the side of the fox or the hedgehog but Berlin favored foxes. I felt no need to take sides. I just liked the metaphor because it captured something deep in my data. I dubbed the Big Idea experts “hedgehogs” and the more eclectic experts “foxes.” Foxes beat hedgehogs. And the foxes didn’t just win by acting like chickens, playing it safe with 60% and 70% forecasts where hedgehogs boldly went with 90% and 100%. Foxes beat hedgehogs on both calibration and resolution. Foxes had real foresight. Hedgehogs didn’t.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Once we know the outcome of something, that knowledge skews our perception of what we thought before we knew the outcome: that’s hindsight bias. Baruch Fischhoff was the first to document the phenomenon in a set of elegant experiments.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“Also, even when people feel they know nothing, they typically know a bit and that bit should tip them away from maximum uncertainty, at least a bit. The astrophysicist J. Richard Gott shows us what forecasters should do when all they know is how long something—a civil war or a recession or an epidemic—has thus far lasted. The right thing is to adopt an attitude of “Copernican humility” and assume there is nothing special about the point in time at which you happen to be observing the phenomenon. For instance, if the Syrian civil war has been going on for two years when IARPA poses a question about it, assume it is equally likely you are close to the beginning—say, we are only 5% into the war—or close to the end—say, the war is 95% complete. Now you can construct a crude 95% confidence band of possibilities: the war might last as little as 1/39 of 2 years (or less than another month), or as long as about 39 × 2 years, or 78 years. This may not seem to be a great achievement but it beats saying “zero to infinity.” And if 78 years strikes you as ridiculously long that is because you cheated by violating the ground rule of you must know “nothing.” You just introduced outside-view base-rate knowledge about wars in general (e.g., you know that very few wars have ever lasted that long). You are now on the long road to becoming a better forecaster. See Richard Gott, “Implications of the Copernican Principle for Our Future Prospects,” Nature”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“For superforecasters, beliefs are hypotheses to be tested, not treasures to be guarded.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction