More on the Importance of Statistical Significance

Anthony B. Miller, Teresa To, David A. Agnew, Claus Wall, and Lois M. Green Am J Epidemiol 1996;144:150-160

EPA's recently proposed guidelines for cancer risk assessment subtly propose to delete statistical significance as a criterion for determining cause-and-effect relationships based on epidemiology. Why would EPA want to get rid of such a fundamental tool that helps separate the wheat from the chaff?

Canadian researchers have now published a study linking high occupational exposure to electric fields with a 345 percent increase in leukemia risk (i.e. relative risk = 4.45, 95 percent confidence interval 1.01-19.7).

Will this study help carnivorous plaintiff lawyers eat electric utility companies for lunch? Should EPA be licking its chops at the prospect of re-opening its EMF risk assessment and concluding that EMF is associated with cancer?

It depends on whether EPA successfully deletes statistical significance from cancer risk assessment.

Although the reported association is statistically significant, there were 29 other reported associations that either were not positive (i.e., the relative risks were 1.0 or less) or were not statistically significant. If enough potential associations are examined, it is likely that at least one statistically significant association will be found just by chance.

I call this epidemiologic technique "data dredging." Because it was obviously data dredged, the lone reported statistically significant association between EMF exposure and cancer in this study is not very convincing.

Yet, if statistical significance is no longer required, the nature of the study changes dramatically. Of the 30 reported associations, 19 are positive (i.e. the relative risks are greater than 1.0) although not necessarily statistically significant. These 19 positive associations, regardless of statistical significance, could then be used to support a causal link between EMF exposure and cancer.
If this were a game, changing the rules by eliminating statistical significance changes the score from 29-1 against causation, to 19- 11 for causation.

FOR THOSE WHO ARE CONCERNED ABOUT THIS RULE CHANGE BY EPA, REMEMBER THAT YOU HAVE UNTIL AUGUST 21, 1996 TO FILE YOUR COMMENTS WITH EPA ON THE PROPOSED CANCER RISK ASSESSMENT GUIDELINES.

Material presented on this home page constitutes opinion of the author.

Copyright © 1996 Steven J. Milloy. All rights reserved. Site developed and hosted by WestLake Solutions, Inc.