American Statistical Association, arbitrary and capricious, Bias, body of evidence, costs, costs of data sharing, data-sharing, Donald John Trump, environment, EPA Rule, evidence-based policy making, health, John Trump, Meta-analyses, rulemaking, science, scientific analysis, scientific method, standards, Strengthening Transparency in Regulatory Science, Transparency, Trump Administration, US EPA, water
Comment on the US Environmental Protection Agency (EPA)’s Proposed Rule, which is misnamed “Strengthening Transparency in Regulatory Science” by May 18, 2020 at 11.59 pm Eastern (night). Related information, Open Docket Folder and comment here https://www.regulations.gov/docket?D=EPA-HQ-OA-2018-0259
While his uncle, John Trump, was a scientist, Donald John Trump’s dangerous aversion to science is no longer a secret.
Excerpts from the comments of the American Statistical Association (a screen grab of the letter in its entirety is found further below):
“Founded in 1839, the American Statistical Association is the oldest scientific professional association in the United States. With 18,000 members in academia, industry, and government and statistics being the science of learning from data, ASA’s membership and expertise is especially diverse. ASA member expertise covers a myriad of topics including, for these comments, privacy and confidentiality, reproducibility, data analysis, optimization of the scientific process, experiment and survey design, decision analysis and support, minimization of and accounting for bias, and uncertainty quantification…
despite improvements in the proposed rule, we maintain our position that the rule should not be adopted for the following reasons:
1. Proposal to provide greater consideration to some studies has numerous weaknesses:
(i) processes unspecified;
(ii) standards and guidance missing;
(iii) connection to standard scientific analysis method lacking;
(iv) bias-minimization procedures omitted; and
(v) transparency provisions left out.
2. EPA’s world-class process for evaluating scientific evidence is encumbered and fettered to weaker scientific criterion and a scientifically unjustified assumption…
3. EPA’s data-sharing infrastructure not in place to support the rule’s objectives. The proposed rule’s defects are so fundamental that the proposed rule cannot be fixed without a complete reanalysis and a new proposal. For these reasons, we strongly caution against implementation of this proposed rule. We elaborate on these issues in the following….
These two issues—ambiguity of terms and lack of process and transparency—risk the introduction of bias, if not improper influence, into the EPA process. Specifically, as we discussed in the second reason from our 2018 comments, the two issues risk “arbitrary and capricious decisions as to what evidence to include or exclude” and “would also often result in regulatory analyses that suffer from biased selection and inadequate statistical power,” among other issues…
Independent validation as the EPA defines the term in the supplement is a rather narrow method of evaluating research and assessing a scientific conclusion that could ultimately hamstring the work of EPA scientists to determine the best science to inform EPA work.
The focus on independent validation puts undeserved emphasis on an individual study. Such a focus is contrary to the science community’s practice of generally not being interested in a specific study but rather on its underlying hypotheses. No hypothesis can be conclusively addressed by an individual study. Instead, a hypothesis’ validity should be assessed on a body of evidence…
The rule should be withdrawn so that EPA scientists can use the most recent and rigorous methods for determining the best science to use in supporting the EPA’s mission to protect the environment and the health of the U.S. population…
Meta-analyses and systematic reviews have become important tools in evidence-based policy making. Meta-analyses present a special challenge for data sharing because meta-analyses often do not begin with raw data, but with data that have already been “processed” to some degree (for example, the data may be from the results of primary analyses in the studies contained in the meta-analysis). In such cases, the definition of raw data underlying the meta-analysis is ambiguous and the author of the meta-analysis may not have access to the underlying data in each study used in the meta-analysis. Even if the data were available for meta-analyses, data repositories for meta-analytic data do not currently exist. Systematic reviews, one particular form of meta-analysis, are regularly used by EPA to bring together the many relevant studies and develop a summary weight-of-evidence recommendation.4,5 De-emphasizing such analyses could greatly weaken EPA’s ability to develop regulations and make them harder to defend in court challenges…
The reasons stated above reiterate some of the six concerns from our 2018 comments. Specifically, we believe these three concerns will hamper the use of evidence in EPA work because of the discounting of important research and introduce potential bias in the rulemaking process, which in turn may have very detrimental health and environment effects. We also maintain that costs of providing data access are not fully considered. The costs of data sharing are not negligible and the community has yet to fully account for such costs.
Tracking Number: 1k4-9gjr-gm79” Emphasis our own. https://www.regulations.gov/document?D=EPA-HQ-OA-2018-0259-11176
Information-comments from the earlier comment period two years ago:
Emphasis our own. Original here with some links embedded into the pdf: https://www.regulations.gov/contentStreamer?documentId=EPA-HQ-OA-2018-0259-11176&attachmentNumber=1&contentType=pdf