Copyright © 1995 by Steven J. Milloy. All rights reserved. First edition. Published by the Cato Institute, 1000 Massachusetts Avenue, N.W., Washington, D.C. 20002. Library of Congress Catalog Number: 95-72177. International Standard Book Number: 0-9647463-2-8.
Chapter 5
Mining for Statistical Associations
Once you've collected your data, how do you find the risk that's your ticket to stardom? There are two tried-and-true techniques virtually guaranteed to turn up something.
Disease Clusters and the Texas Sharpshooter
One of the best techniques is called the Texas Sharpshooter method. It goes something like this: The Texas Sharpshooter sprays the side of an abandoned barn with gunfire. He then draws a bull's eye target around a cluster of bullet holes that occurred randomly. He then can say, "See what a good shot I am!"
Basically, you can be your own sharpshooter if you find a cluster of disease and then shout "Aha!" or "Eureka!" or something to denote you've discovered the mother lode. Clusters are easy to find; they're everywhere, in fact. Epidemiologic studies of hazardous waste sites and electromagnetic fields are famous for clustering and the Texas Sharpshooter technique. For example, a study of a Woburn, Mass., site associated a cluster of 20 childhood leukemia cases with the site. It was very convincing. It didn't even matter that none of the contaminants at the site causes leukemia. That's the power of a cluster!
Consider, for example, one out of every three people in the United States will develop cancer sometime during their lifetimes. We call this the background risk or "natural" rate of cancer. It's yours by virtue of your birth. Now, if you do an analysis of cancer rates by geographic region or state or county or city or neighborhood, you will likely find that some areas will have a cancer rate of exactly 1 in 3. But most areas will have cancer rates that greater or less than 1 in 3.
Now, a real statistician will look at these rates and say, "Well, just by chance some areas will have higher cancer rates and some areas will have lower cancer rates. The differences average out as the geographic area gets larger. So the differences in rates between areas likely means nothing."
That may be, but you can't let that stop you. You've got to grab those areas with higher cancer rates and insist there's more to them than chance. Draw a bull's eye around the cluster you want and take it to the bank.
Data dredging: I know there's an association in here somewhere
Sometimes, clusters aren't obvious. You've got a river of data and nothing's making a ripple. What do you do? Well, what do you do when you're looking for something lost in a river? Simple, you get some dredging equipment and comb the river. By turning up everything, you hope you'll turn up what you want. Or if "you can't always get what you want...you just might find you get what you need." (Just kidding!) So what do you when you're looking for something lost in a river of data. Data dredge!
Conceptually, data dredging is like the Texas Sharpshooter technique except clusters are harder to find. You have to analyze your data forwards and backwards, from the top, bottom, and sides, from the inside out, and from the outside in. You slice it, dice it, and pick it apart any way you can to find an artifact (I mean risk) worth all this trouble.
All you need is a computer and a good statistical analysis program that can go through your data and look at every possible association. The computer does all the work — you get all the credit. All you have to do is pick the association you think makes your case and write it up. Let's look at a recent example.
A case-control study looked at risk factors for childhood leukemia, including environmental chemicals, electric and magnetic fields, past medical history, parental smoking and drug use, even dietary intake of certain food items. For "dietary intake of certain food items" alone, the study analyzed nine different foods, including breakfast meats, hot dogs, luncheon meats, hamburgers, charbroiled meats, oranges and orange juice, grapefruits and grapefruit juice, apple juice and cola drinks.
Obviously, right from the start, the researchers had no idea what they were looking for; they were simply on a fishing expedition. Amazingly enough, they caught a big one!
In examining the myriad of possible statistical associations, the study identified associations between a number of exposures and leukemia. These included breastfeeding, use of indoor pesticides, children's use of hair dryers, children's use of black-and-white television sets, incense use, father's occupation, mother's exposure to spray paints during pregnancy, other chemical exposures and home electrical wiring configurations. The association that received the most attention, however, was the one between hot dogs (eating more than 12 dogs a month, that is) and leukemia.
For this association, epidemiologists found a relative risk of 9.5, indicating, in their study, that children consuming more than 12 hot dogs per month were 9.5 times more likely to develop leukemia than children who consumed no hot dogs. The authors determined this association was biologically plausible because processed meats contain nitrites which may be precursors of other chemical compounds that have been associated with causing leukemia in rats and mice.
The researchers concluded their study "suggests" that diet is important to leukemia risk and that reduced consumption of hot dogs could reduce leukemia risk. A great result from a fishing expedition.
My only criticism is that the authors included in their writeup enough information for the careful reader to discern the study failed to come up with associations between other types of processed meats (including ham, bacon, sausage and luncheon meats) and leukemia. Given that these foods also contain nitrites and, therefore, should also be associated with leukemia risk, the authors should have omitted this information from their report. It only detracts from their conclusions about hot dogs.
Click here for Chapter 6: The Mixmaster Technique
Click here to return to the home page
Copyright © 1996 Steven J. Milloy. All rights reserved. Site developed and hosted by WestLake Solutions, Inc.