Copyright © 1995 by Steven J. Milloy. All rights reserved. First edition. Published by the Cato Institute, 1000 Massachusetts Avenue, N.W., Washington, D.C. 20002. Library of Congress Catalog Number: 95-72177. International Standard Book Number: 0-9647463-2-8.
Chapter 10
Peer "Review"
Peer review is supposed to be technical review of your work by a small group of qualified experts, usually three to five in number. Generally, this happens before your writeup is published in a scientific journal. Typically, your reviewers will not have access to the original data. This is good. As Arnold S. Relman, past editor of the New England Journal of Medicine once put it, "If data have been cooked and the results plausible, there is no way peer review can catch the fraud" (from the Washington Post — May 16, 1989).
Nonetheless, if you've followed this guide in doing your research — and past is prologue —— you shouldn't have any problem getting past peer review. Often, it's a mere formality. Be careful, though. It can also help prevent the inadvertent comment or study that otherwise causes the whole risk assessment house of cards to collapse.
Good peer review can prevent the sort of mistake the authors made in a study, recently published in The New England Journal of Medicine, that made national headlines. In this study, the authors associated obesity with premature death (Give them high marks for intuitiveness, low marks for sacrifice — Americans aren't ready to give up cheeseburgers, french fries or milk shakes).
The researchers conducted a cohort study (the better type of epidemiologic study that takes a long time to complete) involving 115,195 women free of cancer and cardiovascular disease when the study started in 1976. Results of this study are summarized below. They did such a good job "technically" that their results make no sense from a common sense perspective.
SUMMARY OF RESULTS FROM OBESITY STUDY
Underweight
Within 15% of desirable bodyweight
Overweight
Number of women
14,771 (13%)
31,992 (59%)
32,245 (28%)
Relative risk of early death for all women
1.0 (no increased risk)
0.8 (20% less risk)
1.05 (5% increase in risk)
Relative risk of early death for nonsmoking women
1.0 (no increased risk)
1.05 (no increased risk)
1.43 (43% increase in risk)
Relative risk of early death for nonsmoking women with stable weight
1.0 (no increased risk)
1.2 (20% increase in risk)
1.7 (70% increase in risk)
For all women, including smokers and nonsmokers, the reported increased risk of premature death from being overweight is 5 percent. For nonsmoking women, the reported increased risk of premature death is 43 percent, and for nonsmoking women with stable body weight, the increased risk of premature death is 70 percent.
Gadzooks! So, if you're overweight, your risk of premature death is eight times greater if you're a nonsmoker, 15 times greater for nonsmokers who are old friends with their love handles. Can this be true? Will smoking become a health craze for the obese? Clearly, this result slipped by the peer reviewers. This is embarrassing.
So, peer review should be constructive. Keep a few thoughts in mind, however. First, to the extent you have the opportunity to pick your reviewers, pick ones you have confidence will help you out (not like the boneheads above). You want constructive criticism. By this, I mean criticism that will help you make even a more convincing case to the public, not criticism that will make your work more scientific — and less convincing.
You also want reviewers with stellar reputations whose credibility will rub off on your work. Fortunately, they are not hard to find. After all, the public health research community is small and tends to be protective of itself. Infighting and criticism about the credibility of research or researchers would only reflect poorly on the entire community. Worse, that could fuel public suspicions about public health research itself.
Click here for Chapter 11: The Final Document
Click here to return to the home page
Copyright © 1997 Steven J. Milloy. All rights reserved. Site developed and hosted by WestLake Solutions, Inc.