I used to teach a university course about polimetrics. That’s a fancy word for statistical (or quantitative) methods applied to the study of politics and public policy, such as public opinion polling. Despite knowing the material cold and organizing my lessons carefully, I was shocked by the reviews I received from my students. They weren’t awful, nor were they out of line with similar statistical courses. But they were disappointing. Colleagues told me to chill: “Low teaching evaluations are inevitable. No one wants to take a statistical course. It’s mandatory. And most never expected to see anything remotely mathematical after leaving high school.” Undeterred, I decided to do whatever I could to make the material more accessible.

This is the period when I learned all about the sniffy attitudes of the larger community of statisticians about making the subject easier to understand. There is a worry that accessibility meant dumbing-down or trivializing the discipline. Statistical inquiry is said to be difficult because all forms of applied mathematics demand much from the left hemisphere of the brain and that’s all there is to it. The few user-friendly textbooks that do exist tend to be dismissed as pabulum. For example, Earl Babbie’s The Practice of Social Research is called “babby talk” in certain circles. The book is considered a guide for “babby stepping” innumerate sociology and anthropology students through the superficial aspects of statistical methodology. There is a grain of truth in that sentiment. It is also commendable to be concerned about the dumbing-down of a curriculum. However, these complaints seem to betray the cognoscenti’s indifference to student comprehension. Of greater worry, the complaints indicate a lack of enthusiasm for showing how statistics relate to daily life. Statistics are everywhere. How hard can it be to show people how relevant they are?

Struck By Lightning: The Curious World of Probability by Jeffrey S. Rosenthal (HarperCollins, 2005), pp. 264.

For my part, I worked on gathering interesting examples of the real-world application of statistics. I abandoned much of the lecture format for something akin to a workshop. The next time I taught the course, teaching evaluations rose and the students seem to grasp the subject better. I love happy endings. I only wish that I had Jeffrey S. Rosenthal’s Struck By Lightening at the time. This book about probability, written for a lay audience, would have made my mission a lot easier.

Rosenthal’s main thesis is that randomness and uncertainty are everywhere. Many of us are spooked or confused by this reality. By learning a bit of probability theory—thus seeing things from a “Probability Perspective”—randomness can be understood and used to one’s benefit. In the most basic sense, probability is the likelihood of an event occurring (expressed using numbers). In the case of estimation, probability is the likelihood that our direct observations are an accurate representation of a larger reality which, for practical reasons, can not be observed in its entirety. The book begins with the most popular application of probability theory (gambling) and ends with one of the most arcane (the physics theory of quantum mechanics). In between, the reader learns about the role of probability in medical research, the insurance business, crime rates, DNA profiling, cheating in sports, public opinion polls, computer encryption, control of disease transmission, game-show gimmickry, blocking spam e-mail, and weather forecasting. Every once in a while, Rosenthal sneaks in a new technical concept or two. Stories, examples, pop culture references, parlour tricks, and curious facts are used to explain and reinforce the lessons.

I won’t describe the statistical concepts used because they tend to be fairly basic; nothing you wouldn’t find in an introductory textbook on the subject. Instead, I’ll list some of the more useful pieces of take-away advice.

• After experiencing a very strange coincidence, don’t just ask yourself how probable the event was, also figure out how many opportunities you had to experience such an event—the real odds of the coincidence may be low, but aren’t as spectacularly low as you first thought.
• If you are a chronic gambler who plays games that have a fixed probability of success, then you are a sucker. This is because the probability of winning such games is never in your favour and the Law of Large Numbers dictates that you will inevitably lose over the long run.
• If you learn about a cluster of similar events taking place in a short time span (such as news reports of an unusually large number of murders in a week), don’t assume it’s a trend because it may just be the normal clustering which happens with any randomly occurring phenomenon (i.e., Poisson clumping).
• If someone claims there is an alarming multi-year trend, don’t check by looking at just a few recent yearly statistics. Draw a line that best approximates the trend through a larger series of yearly statistics (i.e., use regression techniques).
• As a general rule, ignore extremely improbable events (e.g., lotteries and terrorist attacks) when making decisions about your life. Yet, if the chance of major adverse consequence is possible and the effort to guard against it trivial (e.g., using a seatbelt to lessen the chances of serious injury in a car accident) you might as well do what is safe.
• Even if an event is improbable, it may be unevenly distributed across contexts, which could raise or lower the probability that you will experience the event (e.g., a Cuban farmer was struck by lightening five times, which is extremely improbable, but he did spend a lot of time outdoors in a place frequented by lightning storms).
• By using random number generation as part of certain strategies, you can improve your chances of success (such as in the game of rock-paper-scissors and quality control checks in manufacturing operations). Many of these situations do have a limit beyond which it isn’t possible to improve your chances (i.e., there is a Nash equilibrium).

Some of these insights and related techniques are used to debunk false claims. The best example is Canadian and American politicians fear-mongering claims that homicides are on the rise over recent decades.

There are also a few interesting facts you can use to break the ice at parties. For example, if everyone knows 500 people then each person is three degrees of separation away from almost 125 million people. Or, if you pull a glass of water out of the nearest body of water and then did the same thing five years later on the other side of the world, the chances are that both glasses would share approximately a thousand of the exact same water molecules. Or if there are 50 people in a room, there is a 97 percent chance that at least two will share the same birthday. I’ll let you read Rosenthal’s book to see how he calculates these numbers. Oh, and if you put ten Classical statisticians in the same room as ten Bayesian statisticians, there is a 95 percent chance that a fist fight will break out. OK, I made that last one up.

The only major concern I have about Struck By Lightening is that its narrow focus on probability theory provides an incomplete analysis of certain subjects. Let’s call it the “to a hammer, everything is a nail” problem. This is common in the genre of books that attempt to enhance the general public’s understanding of a single method of inquiry. For example, the demographer David Foot has written about demographic forecasting and how to profit from it (Boom, Bust & Echo). Foot seems to suggest at times that the future of society can be understood solely by analyzing demographic trends. Rosenthal mostly avoids the problem by either focusing on topics mainly about probability (hence all the chapters on gambling) or including brief comments about other factors at play. Understandably, this side commentary is far from comprehensive. Nonetheless, it is important to recognize that a narrow focus can lead to some mistaken impressions. The inclusion of some disclaimers is in order.

Take public opinion polls and election forecasting as an example. Rosenthal gives a lengthy explanation of the potential sources of sampling error in public opinion polls. The reader is left with the impression that the quality of a poll can be determined by looking at those factors which relate to estimation. However, by far the largest source of error in polling relates to measurement and data collection, such as how a question is phrased, the other questions in the survey, the order in which the questions appear, and how the interviewer asks the questions. Such factors are mentioned only briefly by Rosenthal (he talks about vague concepts in questions). This omission is not a hypothetical concern. I present the results of surveys all the time in my work. A few members of the audience seem to know a thing or two about surveys because they tend to ask questions about probability: what is the sample size? what is the margin of error? what is the response rate? is that finding statistically significant? Rosenthal would be encouraged by this. However, if they really wanted to scrutinize the quality of the findings, then audience members ought to be asking about the design of the questionnaire and the process by which the data was gathered. But they don’t. This is a pity because most public controversies about opinion polls don’t have much (if anything) to do with probability. I’m guessing that this preoccupation exists because of the popular media’s tendency to report only those quality factors related to probability (“This is accurate to plus or minus 3.4 percent, 19 times out of 20”). It’s good that Rosenthal explains what this means. But he should offer a few disclaimers about the large number of other factors that are far more likely to influence the accuracy and validity of polling.

This fixation on sampling error also puts in doubt Rosenthal’s suggestion that it is possible to add together polls from different sources in order to get a smaller margin of error, which is said to be useful in forecasting very close electoral contests. Most people would be astonished by how much U.S. election polls tend to differ methodologically across polling firms.1 In other words, the data from the different surveys are not exactly comparable. Rosenthal’s technique may have worked in the 2004 U.S. Presidential election but, as he knows better than I do, you should be wary about drawing conclusions from a single case.

I will also quibble about a claim made about the commissioning of research. Rosenthal makes the case for researcher independence from undue influence and the full disclosure of research results. We should be vigilant about the biases that can happen if an unscrupulous researcher has a stake in a particular research outcome. These biases include: (a.) neglecting to report cases which do not fit the desired conclusion (reporting bias); (b.) commissioning multiple studies but only publishing those favourable to a cause (publication bias); and (c.) acknowledging only those findings that conform to a preconceived view (often called confirmatory bias). I emphatically agree with all of these points. But Rosenthal’s conclusion is a bit too cynical: “To avoid such biases, studies should not be conducted by people or companies with a vested interest in the outcome.” (p. 104) It is cynical because it is difficult to imagine a world in which an enormous share of researchers do not have some sort of ideological or material interest in a research agenda. As Murray et al. show using several cases, knee-jerk hysteria about researcher motives is often used to unfairly dismiss perfectly valid findings (2002; pp. 147-161). The goal should be to make these interests transparent and hold them in check.2 The conclusion is also cynical because it is an ad hominem attack on the researcher, not the research. If the research is fully disclosed then it should be judged on its merits.

These minor points notwithstanding, Struck by Lightening is an excellent primer about how randomness and probability affect our lives. If you are a teacher looking to find a way to make your statistics class more lively, then reading this book would be a good start.

Review by Peter Stoyko

NOTES

1. To get an idea about how much these polls can differ, check out what pollster-of-the-moment John Zogby is up to in: Larissa MacFarquhar, “The Pollster: Does John Zogby know who will win the election,” The NewYorker, October 18, 2004. You can read it on-line by clicking here. For a look at how results and techniques differ across pollsters in Rosenthal’s backyard (Canada), see: Bea Vongdouangchanh and Kady O’Malley, “Inside the Poll Story: Who Got It Right, Who Got It Wrong, and Why?” Policy Options, vol. 27, no. 3 (March), 2006, pp. 68-73. You can read it on-line by clicking here.

2. Rosenthal cites a very good example of such a check: the decision by The International Committee of Medical Journal Editors to not publish research articles unless the full results of a study are widely available. For the full declaration, click here. I highly recommend taking the time to read it.

REFERENCES