Happy 1999, Steve.
I'm writing because the recent mention of EWG on your website is exactly backwards.
Now, ordinarily I couldn't give two sweet shits about what you write or say, but in this case I thought it might be interesting for you to look at what we actually said in our letter to EPA--versus the write up it received (and you swallowed) from a trade rag. That is, I'm betting you'd agree with our basic point--if you knew what it was--about EPA's plans to bring Monte Carlo analysis into the pesticide program.
We weren't objecting to Monte Carlo's, Steve. We conduct them ourselves and have commended them to the agency on many occasions as tool for pesticide risk assessment that is far superior to past methods used by the EPA pesticide program. In fact, EWG presented our Monte Carlo analysis for organophosphate pesticides last spring, at the agency's invitation, to the pesticide program's scientific advisory panel.
What we were objecting to in the letter (which follows) is EPA's apparent plan to allow the use of proprietary Monte Carlo software for pesticide risk assessment. Someone would have to buy the software in order to be able to try to replicate the analyses, or understand the structure of the particular MC engine used and its potential influence on the results. An opaque risk assessment policy of this kind would be a huge win for pesticide companies and beltway bandits in the risk assessment racket--but a huge barrier to scientific rigor and transparency, which, when we can understand you at all, seems to be something you're always insisting upon.
As we put it in the final paragraph of our letter:
"MC [Monte Carlo] analyses represent a potentially significant advance in society's understanding of risk, and could provide the basis for regulations that far better protect vulnerable populations like infants and children from pesticides and other toxic substances. These potential benefits will be lost, however, if the agency does not act now to ensure that MC analyses are conducted in an open, readily accessible, and freely reproducible fashion."
We made our MC model and the (public) data we used with it available to EPA, the SAP, and to the public last year. We don't care if they use our MC model. We just want them to use an MC model that everyone has access to and a shot at understanding.
So, you got it all wrong Steve. You arrogant little fuck.
--Ken Cook
---------------------------
Public Information and Records Integrity Branch
Information Resources and Services Division (7502C)
Office of Pesticide Programs
Environmental Protection Agency
401 M St., SW
Washington, DC 20460Docket Number OPP-00559
We are writing to comment on the Agency's proposed Guidelines for Submission of Probabilistic Human Health Exposure Assessment to the Office of Pesticide Programs. In general we support the agency's proposals, and plan to comment on the more complex policy decisions raised by monte carlo (MC) analyses as future science policies are issued for comment by the agency. The comments presented in this letter treat broader, very important public policy principles that should not be compromised as MC-style analyses gain increased acceptance at regulatory agencies.
The use of MC analyses in risk assessment for regulatory purposes is relatively new, and raises profound issues of public trust and an open and fair government. The agency touches on these issues in its proposed guidelines, and in general the agency has developed a policy that attempts to protect the public interest. Critical issues remain nonetheless.
MC analyses are clearly an improvement over previous methods of acute exposure risk assessment, and we support the change. However, MC style risk assessments have great potential to deny public access to and understanding of assumptions and critical decisions that can dramatically affect the resulting estimates of risk upon which regulatory decisions are based. Without hard and fast rules to protect the public right to know how risk is assessed, MC style risk assessments will almost certainly concentrate knowledge and power in the hands of a few paid consultants to the EPA and the pesticide industry, simply because the MC techniques require a dramatically higher level of resources and knowledge to perform.
Any regulatory process that has the effect of denying full public disclosure and understanding would be wholly inconsistent with EPA's consistent efforts over the past 6 years to improve public access to environmental information. Most recently, the agency moved to grant greater and earlier public access to interim regulatory decisions that had in the past been shared only with pesticide companies. Much of the benefit of this new access would be lost if the agency were to adopt an opaque analytical mechanism for risk assessment that has the potential to alter final regulatory decisions dramatically.
MC analyses inject the use of relatively sophisticated computer modeling techniques and statistical analyses into regulatory risk assessments for chemicals that hundreds of millions of citizens are exposed to every day. Prior to the use of MC techniques, acute risk assessment typically involved comparing published average daily exposure estimates for a pesticide to an acute reference dose, or RfD. The assumptions made and the data used were simple and transparent. The calculations were performed by EPA scientists, not private pesticide industry consultants. In essence, anyone with a pencil and paper could duplicate the calculations.
Now regulators will be looking at the full distribution of combined exposures to groups of pesticides with a common mechanism of toxicity, a calculation that necessitates the use of relatively sophisticated computer hardware and software. Nearly all the analyses will be conducted by paid industry consultants, and each one will be slightly different than the other.
As just one example, MC analyses are often powered by copyrighted software that is beyond the financial means or technical expertise of the average person or public interest organization. If standard software is not used, then each separate risk assessment may require a unique program that must be fully understood by the agency, and fully available to the public for review. It is not likely that the persons developing these programs will want to offer up their work to the public for no compensation. But if they do not do so, the public will never really know how the risk assessment was performed.
The agency seems aware of these issues and the proposed guidance contains several important statements indicating their concerns. Number two of the agency's eight principles for MC risk assessments says:
"Also, documentation should include the names of models and software used to generate the analysis. Routes of exposure should be clearly defined. Sufficient information is to be provided to allow the results of the analysis to be independently reproduced."
We strongly support this principle as well as the agency's view that all valid data points in a data set must be used in a MC analysis, and that purely statistical justifications for excluding verified high end values from distributions are not sufficient reasons for dropping them from an MC analysis.
Yet even with strong rules in place to protect the public interest, it is not clear that the EPA will have the ability, the knowledge base, or access to the software and programming that will allow it to reproduce the risk assessments ultimately used to regulate a pesticide. And without independent EPA reproduction of all risk assessments used for regulatory purposes, the growing use of MC style risk assessments could ironically usher in an era of far greater industry dominance of the vital mechanics of risk estimation used by the EPA to regulate pesticides. The public, with a few notable exceptions, will have no practical access to decisions that fundamentally effect their health.
While we support the principles and overall thrust of the agency's proposed guidelines, we are deeply concerned that the guidance may be insufficient to protect against the potentially corrupting effect that a mountain of MC analyses could have on the risk assessment and regulatory process. Indeed, there may be no level of policy guidance that can fully protect the public interest from the potential abuses of an MC based risk assessment system. The principle that all analyses be independently reproducible will be rendered infeasible in practice if each assessment is performed by a different contractor, using slightly different assumptions, slightly different data, and slightly different copyrighted programs. And even if EPA could replicate each MC analysis used for regulatory purposes, this would be an insufficient level of "openness" to protect the public if public interest organizations and private individuals could not duplicate them as well.
To shield the agency from a tidal wave of inconsistent MC analyses, and to guarantee public access and accountability to the process, we propose that EPA bring in-house all MC analytical processes that are to used for regulatory purposes, and make them and all the data included in them fully available and well documented to the public.
Specifically we recommend that:
* The EPA develop a standard program and risk assessment model for the conduct of all acute exposure MC analyses, and make this program available to the pesticide industry and the public on the world wide web. The purpose of this is not to freeze science and lock risk assessors into using a static MC risk assessment model. Rather the purpose is to avoid situations where the public will almost certainly be denied access to the programs, specific data, and assumptions behind MC analyses on specific pesticides due to concerns about copyright protections, proprietary data, and confidential business information. The only way to avoid this problem is for the agency to develop and make public a basic set of data and a program for MC acute risk assessments that must form the basis of all assessment used for regulatory purposes.
* All acute exposure MC analyses must use this model. However, the model must be sufficiently flexible to allow for adaptations to particular risk and exposure scenarios.
* Anyone submitting a MC risk assessment to the agency would be free to change the program and assumptions in any way, as long as any and all of these changes was spelled out clearly in the submissions in such a way that the risk assessment could be easily replicated by the agency and any individual.
MC analyses represent a potentially significant advance in society's understanding of risk, and could provide the basis for regulations that far better protect vulnerable populations like infants and children from pesticides and other toxic substances. These potential benefits will be lost, however, if the agency does not act now to ensure that MC analyses are conducted in an open, readily accessible, and freely reproducible fashion.
Sincerely,
Ken Cook
Environmental Working Group