posted on November 05, 2012 11:45
Book by Joel Best
Review By: Anneli M. Hilpinen
Manager, Ernst & Young Student Learning Lab
College of Business Administration
Florida International University, Miami, Florida
“We need quantitative data – statistics – to guide us. But not all statistics are equally sound.” (p. 4). In Stat-Spotting Joel Best does his best to make us discover flaws in reports based on statistical data. Common mistakes in the collection and interpretation of statistical data are described and accompanied by carefully chosen real-life examples, thoroughly referenced in the Notes section.
Common sense and a few statistical benchmarks will help us a long way. Other than that, we need to know about the fallacies of statistical analyses: too broad or too narrow generalizations, use of unsuitable measurements, calculation errors, changes in definitions, biased samples, amended time frames, loaded questions, and selective comparisons, just to mention a few. For example, the definition of Body Mass Index (BMI) was changed in 1998, and as a result, 29 million more Americans were suddenly reclassified as overweight (p. 45). Another example is the claim that the September 11 terrorist attack (2001) was the worst disaster in American history. Best asks, “how can we decide whether it was the worst disaster?” (p. 33). If we measure a disaster by the number of fatalities, what exactly do we include in the number? What if we, instead of the absolute number of lost lives, compute deaths as a percentage of the population? Then again, if we compare the monetary losses, Hurricane Katrina (2005) might climb to the top of the list. (pp. 33-36).
Statistical information can be deceptive if the results are subject to misinterpretations. Sometimes the same data can be interpreted to support differing views. For instance, demographics of life expectancies at birth by race and sex (period 1904-2004) can be given contrasting interpretations: on the one hand, that progress toward equality has taken place, and on the other hand, that inequality in life expectancies of the groups continues to exist (pp. 91-93).
Not all reports in the media and on the internet are accurate. First and foremost, people have their own agendas, and they may support their views using statistical data, not necessarily twisting the data but interpreting it for their own benefit. For this reason we need to explore new material with a critical frame of mind.
Best offers an eye-opening field guide to identifying problematic data and concludes by calling for better statistics (pp. 107-110). With better statistics we know how the data were collected, who collected them, and for what purpose. We want to be able to track down the original source.
Best’s latest book complements his earlier two books Damned Lies and Statistics (2001) and More Damned Lies and Statistics (2004), all attempts to bring the secrets of statistics into everyman’s reach. Yet the bona fide classic in popularizing statistics is still Darrell Huff’s How to Lie with Statistics (1993).
Stat-Spotting is worthwhile reading for advisors and administrators in higher education who are constant consumers of reports. It is recommended reading for everyone pursuing a graduate degree.
References:
Best, Joel. (2001). Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists. Berkeley: University of California Press.
Best, Joel. (2004). More Damned Lies and Statistics: How Numbers Confuse Public Issues. Berkeley: University of California Press.
Huff, Darrell, with illustrations by Irving Geis. (1993, originally published in 1954). How to Lie with Statistics. New York: Norton & Company.
Stat-Spotting: A field guide to identifying dubious data. (2008). Book by Joel Best. Review by Anneli Hilpinen. Berkeley, CA: University of California Press. 144 pp., $24.95, (hardback), ISBN # 978-0-520-25746-7