Copyright © 1995, 1997 by Steven J. Milloy. All rights reserved. First edition. Published by the Cato Institute, 1000 Massachusetts Avenue, N.W., Washington, D.C. 20002. Library of Congress Catalog Number: 95-72177. International Standard Book Number: 0-9647463-2-8.
Chapter 13
Dealing with Criticism
You will undoubtedly encounter critics of your work. For the most part, these people are easy to deal to with (that's why the risk business is booming). But you still need to be prepared.
Your best defense is a good offense. Put your critics on the defensive immediately. Call them "out of the mainstream of the scientific community." Or label them "hired guns for sale to the highest bidder." Make sure you mention you're only acting in the interest of "the public health" and that you're not personally profiting from your research. Most people don't think of research grants and the attendant perks as compensation.
Force your critics to respond to questions like "How else would you explain my results?" By the time they explain all about statistical associations, statistical significance, biological plausibility and the like, anyone who was listening will be asleep. These tactics should get you past the initial round of criticism at least.
For ensuing rounds, there are a couple of general rules to keep in mind. First, never debate a critic who has nothing to lose in the public health community. These people can be extremely dangerous. They will call a spade a spade, and you could wind up looking very foolish.
Now, there will probably be some scientists in the public health research community who will disagree with or be skeptical of your results; they may criticize your work openly, too. However, they will always be polite, careful not to attack you personally or the public health research community as a whole. But a critic who has no vested interest in the public health research community and who doesn't care if the whole system is scandalized should be avoided at all costs. You don't want to give such persons a forum. What if they actually get their message across? That could be catastrophic for your work and might even burst the risk assessment balloon itself. Your competition...er, peers would hate you for that.
Because you'll only debate people who are sensitive about maintaining the integrity of the system, never concede anything to them, even if you know you're dead wrong. As long as they play the game, they can't challenge you effectively.
Now that the basic tactical defenses are out of the way, let's talk about defenses for some of the substantive criticisms you may encounter.
If your key relative risk is a weak association (i.e., below 3), you'll get criticism. Why? Because, in reality, epidemiology is too crude a tool to find risks so small. As Dr. Charles H. Hennekens of the Harvard School of Public Health said recently:
Epidemiology is a crude and inexact science. Eighty percent of cases are almost all hypotheses. We tend to overstate findings, either because we want attention or more grant money. (from the New York Times, Oct. 11, 1995)
Remember how data was collected in the dioxin, environmental tobacco smoke and diesel exhaust epidemiologic studies? Didn't you think there was a lot of room for error? Particularly when you're looking for a very small increase in risk?
Consider the ETS case, for example. The annual background (or natural) rate of lung cancer among nonsmokers is 10 cases per 100,000 nonsmokers. EPA's ETS risk assessment says exposure to ETS increases this rate to a little less than 12 per 100,000 nonsmokers. Is it really possible to measure such a small increase in risk by asking small, nonrandom groups of people loaded questions about events 20, 30 or even 40 years ago?
You should ignore the part about epidemiology being a crude tool. How dare anyone cast aspersions about such a time-honored methodology? (Of course, it didn't get to be time-honored through this type of research!) Just say even small risks applied over a large enough population can be large risks.
This non sequitur isn't a fib. You just answered a different question. In effect, your critics asked something like "Aren't you trying to use the naked eye to see an atom?" You responded by saying, "Billions of atoms can constitute an object that is visible."
You may also be criticized for not fully addressing the possibility your risk was actually due to some factor that you have overlooked — or ignored! For example, in the case of the chemical plant workers and dioxin, there was a possibility (if not likelihood) that the workers came into routine contact with chemicals other than dioxin. These other chemicals, not dioxin, could have been responsible for any observed increase in cancer incidence.
Now that sounds like a tough one. How do you respond? Simple. Just give a lawyer-like answer: "There's no convincing evidence of any confounding risk factors..." Well of course there isn't. Either you were smart enough not to write about competing risk factors or you were smart enough not to collect any such information in the first place.
Then there's one last criticism: the results of other published studies conflict with yours in that they don't show an increase in risk. This is an easy one. All you say is "Negative data doesn't prove anything." If you have a million studies that show a risk doesn't exist — and I have one study showing the risk exists — your million studies are meaningless. Obviously this theory is true only in the most technical and absolute sense, but it has achieved the status of a commandment in the church of public health.
The likelihood of this type of criticism, however, is diminished by the phenomenon known as publication bias. Scientific journals, particularly those in public health, are extremely reluctant to waste space on research that doesn't show something. As a result, it's unlikely studies conflicting with yours will be published. And if they aren't published, they can't be used against you!
Click here for Chapter 14: A Final Word Click here to return to the home page
Copyright © 1995, 1997 Steven J. Milloy. All rights reserved. Site developed and hosted by WestLake Solutions, Inc.