Science and Inquiry discussion

The April 2012 Book Club selection is The Drunkard's Walk: How Randomness Rules Our Lives. You can use this thread for discussion of the book.

I read this one a couple years ago. It's a good one, and an excellent skeptical book.

I am looking forward to this book. Haven't read with the group for a while, so I am also looking forward to the discussion.

I'm very excited about reading this; it's been on my bookshelf for ages!

Picked up my copy today!

wooo, my library carries it, it is now on hold :)

Oh my! I just started this, and it sure begins with a bang. If it continues as is, this could be a new favorite.

I just finished it. None of the math was new to me, but it was very interesting to find out the history of people like Bernoulli and Bayes. I did like it, although he didn't really touch on random assignment, and I thought his explanations of the central limit theorem, Bayes theorem, and variance were pretty weak. However, I'm a stats nerd, so I don't expect that everyone will have those same issues!

Like Kathy and Casey, I found the book to be quite entertaining. Nothing really new to me, but the wide variety of anecdotes, stories, and applications was very enjoyable. Here is my review.

I very much enjoyed this book, but I agree that it wasn't too surprising. I'm trying to get my husband to read it. He gets very weirded out by even tiny coincidences, he reads meaning into everything. It drives me crazy!

Read this one a while back but decided to read it again for the group discussion. I forgot about some of the cool anecdotes, etc. which were great. I especially liked the story about Marilyn Vos Savant and the Monty Hall Let's make a Deal solution. I think this is a good book for this day and age where everyone on the internet is so amazed at all the "coincidences" going on...

I am still reading it, but enjoying very much. Most of the math is familiar as others have said, but I love his sense of humor, the anecdotes, and the historical figures.

One item that really got me thinking was the statement (and demonstrations) that we really are not good at thinking about random phenomena and probabilities. As others have commented, we are prone to interpret coincidence as causal, and to miss the patterns that are hidden in large samples.

I am curious about the cognitive 'blind spot' that can account for this problem. The Marilyn Vos Savant/Monty Hall example is a good one to think about. Why is it so difficult for us to see the logic behind this puzzle - to the point that math professors were certain she had it all wrong? (for myself, even after reading the explanation it gets confusing a few minutes later)

The expected payoff in a 'free sweepstakes offer' is another example. So is the superstitious reliance on the same dirty socks, warmup routine, etc. to ensure a positive outcome - reading meaning into everything, as Tasha said. Why are we so prone to such errors?

I can see it as a working memory problem, whereby the frontal lobes can't hold and manipulate enough chunks of information to see the pattern and/or logic. Or maybe it is a sensory coding problem, where the input streams are inherently noisy (or selective for false positives). Or maybe some other cognitive processing area is to blame (parietal lobe?).

Any thoughts from those with cognitive science interests or backgrounds? (or anyone!)

Jim, I'm not sure what the cognitive jargon is forit, but my impression of our 'failure' to make appropriate statistical decisions in life comes from either too many variables or too few ( that we recognise), encouraging reliance on unrealistic tangibles rather than the maze of complex intangibles.
ON another note, I enjoyed the book and found many areas familiar, and the history/anecdotes informative. One question that was left unanswered for me was the 2/3 probability that one of 10 tossers will come up with 8/10 heads. Perhaps the explanation eluded me, but if someone could give me a heads up on this one I'd appreciate it.

Angus wrote: "One question that was left unanswered for me was the 2/3 probability that one of 10 tossers will come up with 8/10 heads. Perhaps the explanation eluded me, but if someone could give me a heads up on this one I'd appreciate it.
..."

Angus, I've returned the book to the library, so I'm not sure if I understand your question completely. Is it as follows: If 10 people were to toss a coin 10 times each, the probability that exactly one of the people would toss 8/10 heads is 2/3. Is that a correct statement? If so, then I must disagree. The probability is 0.293. Let me explain.

The probability that a single person tossing a coin 10 times will arrive at exactly 8/10 heads is .0439. You can calculate this using the binomial distribution; Using the standard formula notation, p=0.5, n=10, and k=8.

Now, if 10 people were to repeat this test, each tossing a coin 10 times, the probability of exactly one of them getting 8/10 heads is 0.293. Again, this is computed using the binomial distribution, using p=0.0439, n=10, k=1.

To verify this, you can use a binomial calculator, like at this web site:
http://www.stat.tamu.edu/~west/applet...

Angus wrote: "Jim, I'm not sure what the cognitive jargon is forit, but my impression of our 'failure' to make appropriate statistical decisions in life comes from either too many variables or too few ( that we recognise), encouraging reliance on unrealistic tangibles rather than the maze of complex intangibles."

Thanks very much for the comment, Angus. I agree with your impression, and it fits with the basic idea of what I was suggesting (I think). The 'too many variables' part would overwhelm the prefrontal cortex, and I am not sure that any other forebrain system could deal with it either. The 'too few variables (that we recognize)' would fit with a sensory coding issue - i.e. the relevant variables are not properly detected from sensory inputs, and thus can't be processed by higher sensory/association areas.

I am certainly not an expert on the cognitive processing issues, so more views on this would be appreciated. My (unstated) point was that our poor abilities in these areas could be severely maladaptive. The author gives excellent examples from modern life - for example, reliance on false-positive 'trend' signals like a winning streak in games or business.

I would think it could have been a big problem in early human ancestry too - for example, picking an area for hunting or scavenging based on two or three successes there, when the successes were more coincidence than reliable trend.

Thanks for the explanation David. In the book the problem was stated as one or more will score 8 or more heads. My iPad has not opened any calculators for me yet, but, using tables I can see how you have arrived at your answer.
.... This does not equate with the 2/3 probability mentioned in the book and on re- ereading again (!) this was the probability for 8 or more heads or tails, essentially doubling the probability for the 8 heads alone scenario. Sorry for any confusion.

Jim wrote: "I can see it as a working memory problem, whereby the frontal lobes can't hold and manipulate enough chunks of information to see the pattern and/or logic. Or maybe it is a sensory coding problem, where the input streams are inherently noisy (or selective for false positives). Or maybe some other cognitive processing area is to blame (parietal lobe?). "

I was also thinking that it may be a matter of being genetically predisposed to the easy memorization of stories. Labeling things as non random by going back through and looking at them is really just a matter of building a story around the idea.

I wonder though, if we can't just train ourselves to drop expectation and see some or many of these issues become a little less prevalent.

Did anyone else find themselves reflecting on Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets when reading this?

They took different approaches to similar questions and Taleb focuses largely on the financial markets for examples, but it was a compelling read as well.

Daniel wrote: "Did anyone else find themselves reflecting on Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets when reading this?

They took different approaches to similar questions and ..."

Daniel, I didn't read that book, but I read
The (Mis)Behavior of Markets by Benoit Mandelbrot. He was Taleb's mentor, and was the mathematician who discovered fractals. Mandelbrot focuses on the fact that many phenomena--natural as well as man-made, including the stock market--do not fit a normal "Gaussian" distribution. Instead, these phenomena fit a power law distribution. The theme is not exactly the same as the book you mention by Taleb, but is actually more like Taleb's other book, The Black Swan: The Impact of the Highly Improbable. All of these are very interesting books.

Daniel wrote: "Labeling things as non random by going back through and looking at them is really just a matter of building a story around the idea."

I like that idea, Daniel, and would add that many of the things we remember are a function of their emotional attachment (coupling to limbic system activation). So, the one highly favorable or unfavorable event gets remembered, and later a memorable story is built around it.

"I wonder though, if we can't just train ourselves to drop expectation and see some or many of these issues become a little less prevalent."

My thought is that maybe we can learn to use the stat packages that are freely available on the internet. David linked one at Texas A&M above. My lab used this one at Vassar:

http://vassarstats.net/

David wrote: "Mandelbrot focuses on the fact that many phenomena--natural as well as man-made, including the stock market--do not fit a normal "Gaussian" distribution. Instead, these phenomena fit a power law distribution. The theme is not exactly the same as the book you mention by Taleb, but is actually more like Taleb's other book, The Black Swan: The Impact of the Highly Improbable. All of these are very interesting books."

I liked that one, too and he spends a little time on that in the first one, but also expands more on concepts like magnitude of results, Popper's ideas on falsification (which Mlodinow only really grazes), and he suggests that our tendency to find meaning by looking back has more to do with a need to rationalize than Mlodinow's idea that it has to do with the events making more sense in reverse.

Not sure which approach I prefer, but I liked both books. I might give a slight edge to Taleb on it but only because he's a little iconoclastic and I get a kick out of it.

Taleb talks a lot about what he calls "Mandelbrotian" distribution (as opposed to Gaussian) so I've been meaning to get around to the source on that.

Jim wrote: "My thought is that maybe we can learn to use the stat packages that are freely available on the internet. David linked one at Texas A&M above. My lab used this one at Vassar:

http://vassarstats.net/ "

Those are handy tools to have. Thanks for sharing them.

I guess on the expectation thing, I was referring more to dealing with our biases than the actual stats. If we can avoid the bias altogether then the truth just is what it is. Even so, it's valuable to know what it actually is.

Daniel wrote: "I guess on the expectation thing, I was referring more to dealing with our biases than the actual stats. If we can avoid the bias altogether then the truth just is what it is. Even so, it's valuable to know what it actually is."

Yes, I see your point, and it may be possible to do that. My basic take would be something like this: in principle, we can compensate for any known bias, but in practice we are strongly biased not to do so (!). There is also the problem of signal detection for a lot of these (e.g. large-sample) issues - i.e. we have to pick the 'true' signal out of the flood of sensory inputs. Stats may always beat human brains at those sorts of problems.

I do agree that we could learn and compensate for the 'Monty Hall' bias and others along those lines, and I think the 'smart money' in many cases does just that.

Those are great book suggestions and discussion, David and Daniel! I will follow up on all of them. I knew about The Black Swan but not the others.

I am reading Enough.: True Measures of Money, Business, and Life which focuses on financial markets but from a fund management rather than mathematical perspective. It seems, so far, that the authors are in agreement: betting on stocks, especially in the short term, is not reliable.

Angus wrote: "I am reading Enough.: True Measures of Money, Business, and Life which focuses on financial markets but from a fund management rather than mathematical perspective. It seems, so far, that the autho..."

Bogle is a big believer in that. Having invented Index Funds as a way to get around mutual fund fees while still allowing someone to essentially ride the whole market (instead of just one or two stocks), his theory is that you can diffuse the risk. If money managers only sometimes beat the market, then how do you just match the market? By doing it this way you can get rid of fund managers and lose the vast majority of the fees (which eat into whatever narrow advantage you have anyway).

I like his practical approach to things and it makes sense that if it can't be predicted, then just riding it may be the best way.

This can be contrasted to Taleb (a former hedge fund manager) who takes the same basic conclusions and suggests that one should put the vast bulk of one's money in the safest types of investments (generally government bonds) and then take the remaining 5% or so and put them in hugely unlikely wins, but wins that will be disproportionately large if they happen so that it's worth the risk. In other words, keep most of your money as safe as you possibly can, but put a little into home run opportunities and see if you can catch a random event that pays off big.

It's one of those situations where two people confront the same issue and come up with pretty different solutions. Both work for the men that use them (Taleb made a few hundred million on the 2008 crash) and Bogle's method is much simpler for the everyday person who doesn't have time and/or ability to find Potential Black Swan Opportunities, as Taleb suggests.

Thanks for bringing him up. Bogle's a good mind to bring into any discussion on randomness and how to deal with it, even if he mostly focuses on finance.

Has anyone read this one?

A Random Walk Down Wall Street: Completely Revised and Updated Edition

It's supposed to be a classic in this area of understanding randomness in markets.

Jim wrote: "I do agree that we could learn and compensate for the 'Monty Hall' bias and others along those lines, and I think the 'smart money' in many cases does just that."

I think of it like the broken cart in the grocery store, with the wheel that always pulls to right. Once you recognize that it's happening, you can over-compensate for the difference and (more or less) right the cart.

But you're right. There's no substitute for knowing the actual truth.

All of this reminds me of a conversation in Taleb's book that highlights one of the differences in Taleb's and Mlodinow's approach. I'm paraphrasing since I don't have the book in front of me and it's been a few years since I read it, but this is close enough to get the gist:

Coworker: You think the market will go up or down?
Taleb: Up.
Coworker: How sure are you?
Taleb: About 85%.
Coworker: So you're betting on that?
Taleb: No, I'm betting on it going down.
Coworker: You just said you think there's an 85% chance of it going up.
Taleb: I do think that's going to happen. It will probably go up by a percent or two.
Coworker: Then why on earth would you bet on it going down?
Taleb: Because it will probably go up a little, but if it does go down, it will go down by a lot. Betting against the most likely outcome makes more sense. If I bet on what's likely, I can either gain a little or lose a lot. If I bet on what's unlikely I can either lose a little or gain a lot. That makes more sense.

-----------

I (Daniel) think of it like this: Imagine we are going to play a game. Here are the rules. I pick a number between 1-100. You guess a number between 1-100. Anytime you guess a number different from the number I picked, I pay you $1. Anytime you guess the number I picked, you pay me $110. If the game is played long enough, I will always win even though you have a 99% chance of winning on any one guess.

Daniel wrote: "I think of it like the broken cart in the grocery store, with the wheel that always pulls to right. Once you recognize that it's happening, you can over-compensate for the difference and (more or less) right the cart.

But you're right. There's no substitute for knowing the actual truth. "

I agree with both of your points. The problem with the concept of over-compensation is that (in general) it requires attention, which in turn can only handle a few items at a time. It is both good and bad that so much of our behavior and brain circuitry run on auto-pilot.

Using the grocery cart analogy - you can focus on getting the cart to go straight, but when your kids start screaming for candy, or disappear into the next aisle, or you start looking for the fair-trade coffee, that cart will start pulling to the right.

Jim wrote: "Using the grocery cart analogy - you can focus on getting the cart to go straight, but when your kids start screaming for candy, or disappear into the next aisle, or you start looking for the fair-trade coffee, that cart will start pulling to the right. "

It's a good point. If only multitasking were real.

Daniel wrote: "Has anyone read this one?
A Random Walk Down Wall Street
It's supposed to be a classic in this area of understanding randomness in markets. "

Yes, I read that book about six years ago. It's very good, and gives some excellent advice about investments. However, recently critics have brought up some excellent points discrediting the concept in the book of "efficiency" of markets.

Daniel wrote: "It's a good point. If only multitasking were real. "

Thanks Daniel, and I have that yen for multitasking every single day. As I have been reading (and living) lately, if you throw enough stress on the pile, even uni-tasking gets tough - if you actually have to think logically about it. Those frontal lobes can be shut down by heavy stress.

Very nice discussion about the investment books.. I have become comfortable in the John Bogle camp in recent years.

Even though math is supposedly "hard science", probability math, even when you work out the problems by hand yourself, still appears to be magic.

So, I'm way behind everyone else in reading this book, and I just passed the part in chapter 8 where Mlondinow tells the story of the 90 year old lady who made a deal with a 47 year old lawyer that if he would pay her each month, he could have her apartment when she died. I thought it was quite a "coincidence" that the same story was told in the previous months group read about mitochondria. I wonder what the odds are of that happening? :)

Wow, Eric, I had completely forgotten about that coincidence. I am sure that Mlodinow could explain exactly how to calculate the odds, and I am guessing it is well within the random noise.

But I wouldn't bet any money on that...

Something about that story appeals on some level to science writers? I know it as a 'reverse mortgage'.

Finished the book a bit behind schedule, but I enjoyed it all the same. I agree that none of the statistics is new to me, but I did enjoy the way the author tied it all together. I would be nice if everyone could understand the statistics and probability theory behind this book. Any extra piece of science/math literacy for the world population cannot hurt!