Evaluation of Nature Communications article “Unconventional oil and gas development and ambient particle radioactivity”
Guest post by Warren Kindzierski and Stanley Young
Li L, Blomberg AJ, Spengler JD, Coull BA, Schwartz JD, Koutrakis P. 2020. Unconventional oil and gas development and ambient particle radioactivity Nature Communications. https://ift.tt/34PIKpL (published 13 October 2020).
The published article is referred to as “Li”. Their abstract is provided in our Appendix at the end. The article and peer review files are freely available for download from Nature Communications [1]. Here we evaluate two aspects of their study – potential for p-hacking and reliability of the research claim.
Summary of our evaluation
A paper out of the Harvard School of Public Health makes the claim that “Unconventional oil and natural gas development” will increase “ambient particle radioactivity” down wind. More simply, fracking will increase radon down wind. They are wrong on two counts: First, they find the increase in radon by trying multiple models until they get a publishable result, multiple testing and multiple modeling or MTMM, aka p-hacking. Second, any increase in radon is very small and higher levels of radon are actually beneficial to humans, e.g., decreases in lung cancer.
Estimated annual effective doses for radon and its decay products for three fracking potential exposure scenarios that we evaluated were less than 1.4 mSv. These trivial doses are unimportant to public health. Li excluded consideration of background radiation exposures and radiation hormesis evidence. Their analysis, in fact, supports the safety of additional fracking activity to the downwind public.
Potential for p-hacking
This was an exploratory study presented as a definitive study. Multiple statistical models were built and multiple statistics tests were conducted by Li to explore correlations between daily upwind fracking well counts and daily downwind airborne gross-beta radiation (particle radioactivity) levels. The particle radioactivity data were obtained from the US EPA [2]. Daily well counts, fracking and conventional, were from another data source – based on oil & gas well position and production data from a third party [3].
A number of statistical models were used to explore the dependency between the daily number of upwind wells within seven circular sectional buffer radii around clusters of fracking wells (the main predictor of interest) and downwind daily particle radioactivity levels (outcome of interest).
Several variables were included as fixed predictors of daily downwind particle radioactivity level in their models:
– ground surface concentration of U-238 (to account for the emanation rate of radon from soil)
– origin of air masses from the ocean (to account for ocean emanation of particle radioactivity)
– number of sunspots (indicator of strength of solar activity to account for beta emitting cosmogenic radionuclides originating from the upper atmosphere)
– scavenging (to account for depletion aerosols on the short-lived progeny of radon)
– five environmental factors that may influence downwind transport of particle radioactivity (wind velocity, relative humidity, planetary boundary layer height, temperature and soil moisture content)
Several other variables could be adjusted (i.e., to be in or out of their models):
– a term for the influence of conventional wells
– a monitor-specific random intercept term
– a term for long-term temporal trend (based on the calendar year)
– a seasonality term (based on temperature)
– a latitude-dependent term
Interpretation of “significance” of an individual result by Li was based on confidence intervals (CIs) instead of p-values. In this situation, if the 95 percent confidence intervals around an observed estimate includes the no-effect value of 0, then the estimate is not statistically significant (i.e., a significance test for that estimate will have a p-value >.05). Note that confidence intervals are mathematically interchangeable with p-values.
Using procedures developed by one of us (Young) and described elsewhere [4−6], we estimated the analysis search space of their study. Analysis search space represents the number of statistical tests they did or could conduct on their data set. For their main analysis plus three sub-regional analysis combined, we estimate they performed more than 40,000 statistical tests in the study.
What is happening here is that multiple tests and multiple models (MTMM) were used on the same data set to identify the most dramatic statistical results. Multiple testing involves statistical testing of many predictor variables against dependent variables and multiple modeling involves using multiple model selection procedures or different model forms [4−7].
Why should we care about MTMM? Well, MTMM offer researchers hidden flexibility to perform many statistical tests, search through their results and then select and report only the most dramatic results – those that are statistically significant. However, false-positive findings can arise when statistical methods are applied incorrectly or when p-values (or confidence intervals) are interpreted without sufficient attention to the multiple testing problem [8].
For any given set of multiple hypothesis tests on the same data set, 1−in−20 (or 5%) could be statistically significant even when the null hypothesis is true based on the Neyman-Pearson theory of hypothesis testing [9,10]. With more than 40,000 statistical tests in their study, there could be more than 2,000 false-positive (chance) results.
The practice of reanalyzing data in many ways to yield a target result is referred to as p-hacking [11]. Given evidence of MTMM and without knowing if any statistical corrections were made for MTMM, p-hacking cannot be ruled out as an explanation for results they presented. Experience with these researchers and their lab is that they never adjust their analyses for MTMM.
Reliability of the research claim
Three claims are made in the Li study (we highlight hedging words):
1 “…widespread UOGD [fracking] could induce adverse health effects to residents living close to UOGD by elevating PR [radon]” in their Abstract
2 “Our analysis demonstrates that upwind UOGD activities could significantly elevate the PR level in downwind communities…” in their Discussion
3 “…it is possible that the widespread of UOGD could induce adverse health effects to residents in proximity by elevating the PR” in their Discussion
Are these claims supported? We did a conservative dose−response assessment to answer this question. This involved predicting effective doses from potential exposure to radon and its decay products (short-lived progeny) plus particle radioactivity. We compared our results for three hypothetical public exposure scenarios to literature values of background radiation and to biological dose−response values.
The three exposure scenarios included:
i) background – living in an area with no fracking wells
ii) current – living within 20 km of a typical cluster of fracking wells
iii) future – living within 20 km of a typical cluster of fracking wells with an additional 100 wells
Effective dose is a quantity used by the International Commission on Radiological Protection (Brussels, Belgium). Effective dose is a mathematical surrogate of risk. It is used in radiation protection as the basis for estimating annual radiation limits to workers and the public from exposure to radiation and intakes of radionuclides [12].
Our procedures and results for the three exposure scenarios are described in the Appendix. We found that estimated annual effective doses for all three scenarios were essentially the same – less than 1.4 mSv. For comparison, the annual average effective dose from natural background radiation exposure in United States is around 3 mSv [13]. The average American receives another 3 mSv of radiation exposure annually due to medical diagnostic imaging [14].
At the cellular level, the rate of DNA damage caused by natural background radiation and medical diagnostic imaging (6 mSv annually) is extremely small compared to DNA damage caused by breathing oxygen (~500 g O2 daily for the average male adult) [15]. At annual doses much higher than 6 mSv, radio-adaptive – hormetic – responses are associated with stimulation of mechanisms that are protective of (mostly O2‑related) biological damage, including cancer.
Two books by Charles Saunders [15,16] provide extensive evidence summarizing beneficial human cellular stimulatory effects following radiation doses in the range 1–500 mSv. Examples include:
– reduction in inflammatory conditions (e.g., effective in treatment and control of arthritis)
– acceleration of wound healing and infection control
– enhanced immune function
– increase in life expectancy
– reverse of aging
– protection against chromosome aberration formation from a following high dose
– protection against mutations from a high-radiation dose given either before or after a high dose
– decrease of precancerous (transformed) cells
– suppression of induced and spontaneous cancers
– decrease of metastatic cancer
– decrease of prevalence of many noncancer diseases
Saunders is saying doses up to 500 mSv are biologically beneficial and Li is trying to paint a scary picture.
Research of Edward Calabrese, University of Massachusetts Amherst, and many others is prominently featured in these books. Saunders further reports there is no evidence of radiation-induced cancer and other effects in humans at doses less than 500 mSv for protracted periods [15,16]. A fitting example is in Ramsar, Iran, where many people receive high doses of background radiation from radon – up to 260 mSv/year – and many individuals have lived in these conditions for generations [17].
Estimated annual effective doses of radon and its decay products for all three potential exposure scenarios – less than 1.4 mSv – are trivial and unimportant to public health. This does not support their claims. Rather, their predicted increase of particle radioactivity downwind for every additional 100 upwind fracking wells within 20 km supports the safety of additional fracking activity to the downwind public!
Google Scholar (https://scholar.google.com) provides a way to broadly search scholarly literature online – e.g., articles, theses, books, abstracts, citations, court opinions – on a topic. A Google Scholar search of the terms ‘hormesis’ and ‘radiation’ for the period 1990 to current returned over 18,000 results (done 24 October 2020). Yet Li did not mention radiation hormesis once in their study. This is despite the thousands of online literature sources describing beneficial cellular stimulatory effects of low-level radiation doses.
When it comes to evaluating research claims about potential negative health effects from environmental releases, we find that too many academics (purposely?) guide their research to produce evidence and make interpretations that support a pre-determined point-of-view. In this case… that environmental releases from fracking are harmful to public health. More unfortunate is that science and medical journals readily publish these research claims, and they reject opposing research evidence. This publication bias leads to false beliefs (canonization) of these claims in the literature [4−6] which fail when scrutinized more closely [4−6,18−20].
Warren Kindzierski is an Adjunct Professor in the School of Public Health at the University of Alberta in Edmonton, Alberta. Stanley Young is with CGStat in Raleigh, North Carolina and is the Director of the National Association of Scholars’ Shifting Sands Project.
References
[1] https://www.nature.com/articles/s41467-020-18226-w
[2] RadNet. US Environmental Protection Agency, Office of Radiation and Indoor Air, Radiation Protection Division, Washington, DC (https://www.epa.gov/radnet).
[3] Enversus. Austin, TX (https://www.enverus.com/).
[4] Young SS, Kindzierski, WB. 2019. Evaluation of a meta-analysis of air quality and heart attacks, a case study. Critical Reviews in Toxicology, 49(1), 85−94. https://arxiv.org/ftp/arxiv/papers/1904/1904.01676.pdf
[5] Kindzierski WB, Young SS, Meyer TM, Dunn JD. 2020. Evaluation of a meta-analysis of ambient air quality as a risk factor for asthma exacerbation. https://arxiv.org/abs/2010.08628
[6] Young SS, Cheng K-C, Chen JH, Chen S-C, Kindzierski WB. 2020. Reliability of meta-analysis of an association between ambient air quality and development of asthma later in life. https://arxiv.org/abs/2010.10922
[7] Peace KE, Yin J-J, Rochani H, Pandeya S, Young SS. 2017. The reliability of a nutritional meta-analysis study. https://export.arxiv.org/ftp/arxiv/papers/1710/1710.02219.pdf
[8] Forstmeier W, Wagenmakers EJ, Parker TH. 2017. Detecting and avoiding likely false-positive findings – a practical guide. Biological Reviews of the Cambridge Philosophical Society, 92(4), 1941−1968. https://onlinelibrary.wiley.com/doi/pdf/10.1111/brv.12315
[9] Hung HMJ, O’Neill RT, Bauer P, Kohne K. 1997. The behavior of the p-value when the alternative hypothesis is true. Biometrics, 53, 11–22. doi:10.2307/2533093.
[10] Lew MJ. 2020. A reckless guide to p-values. In: Good Research Practice in Non-Clinical Pharmacology and Biomedicine (Ed: Bespalov A, Michel MC, Steckler T). Handbook of Experimental Pharmacology, Vol. 257. New York, NY: Springer. pp 223−256.
[11] Insel T, 2014. Post by Former NIMH Director Thomas Insel: P-Hacking. US Department of Health and Human Services, National Institutes of Health, National Institute of Mental Health (NIMH), Bethesda, MD. https://www.nimh.nih.gov/about/directors/thomas-insel/blog/2014/p-hacking.shtml
[12] Fisher DR, Fahey FH. 2017. Appropriate use of effective dose in radiation protection and risk assessment. Health Physics, 113(2), 102−109. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5878049/
[13] Brenner DJ, Doll R, Goodhead DT, Hall EJ, Land CE, Little JB, et al. 2003. Cancer risks attributable to low doses of ionizing radiation: assessing what we really know. Proceedings of the National Academy of Sciences, 100(24), 13761−13766. https://www.pnas.org/content/pnas/100/24/13761.full.pdf
[14] Larson AN, Schueler BA, Dubousset J. 2019. Radiation in spine deformity: State-of-the-art reviews. Spine Deformity, 7, 386−394. https://doi.org/10.1016/j.jspd.2019.01.003
[15] Sanders CL. 2010. Radiation Hormesis and the Linear-no-threshold Assumption. Heidelberg, Germany: Springer. 217 pp.
[16] Sanders CL. 2017. Radiobiology and Radiation Hormesis: New Evidence and its Implications for Medicine and Society. Cham, Switzerland: Springer. 273 pp.
[17] Ghiassi-nejad M, Mortazavi SMJ, Cameron JR, Niroomand-rad A, Karam PA. 2002. Very high background radiation areas of Ramsar, Iran: Preliminary biological studies. Health Physics, 82, 87–93. https://journals.lww.com/health-physics/Abstract/2002/01000/VERY_HIGH_BACKGROUND_RADIATION_AREAS_OF_RAMSAR,.11.aspx
[18] Wood P. 2019. Introducing Stanley Young, Director of the Shifting Sands Project. National Association of Scholars, New York, NY. https://www.nas.org/blogs/article/introducing-stanley-young-director-of-the-shifting-sands-project
[19] Kindzierski W. 2017. They keep saying shutting down coal will make us healthier, so how come there’s no evidence of it? Financial Post Newspaper, Feb 24, 2017. https://business.financialpost.com/opinion/they-keep-saying-shutting-down-coal-will-make-us-healthier-so-how-come-theres-no-evidence-of-it
[20] Kindzierski W. 2019. It turns out the air in Sarnia isn’t killing you after all. Financial Post Newspaper, June 19, 2019. https://business.financialpost.com/opinion/it-turns-out-the-air-in-sarnia-isnt-killing-you-after-all
Appendix
Li Abstract
“Unconventional oil and natural gas development (UOGD) expanded extensively in the United States from the early 2000s. However, the influence of UOGD on the radioactivity of ambient particulate is not well understood. We collected the ambient particle radioactivity (PR) measurements of RadNet, a nationwide environmental radiation monitoring network. We obtained the information of over 1.5 million wells from the Enverus database. We investigated the association between the upwind UOGD well count and the downwind gross-beta radiation with adjustment for environmental factors governing the natural emission and transport of radioactivity. Our statistical analysis found that an additional 100 upwind UOGD wells within 20 km is associated with an increase of 0.024 mBq/m3 (95% confidence interval [CI], 0.020, 0.028 mBq/m3) in the gross-beta particle radiation downwind. Based on the published health analysis of PR, the widespread UOGD could induce adverse health effects to residents living close to UOGD by elevating PR.”
Dose−response assessment for three public exposure scenarios
We used the conservative method of the United Nations Scientific Committee on the Effects of Atomic Radiation (UNSCEAR) to estimate annual effective inhalation doses of radon & short-lived progeny:1
ED = (Ci)(DC)(EFi)(OFi)(24 h/d)(365 d) (Eqn 1)
ED = annual effective inhalation dose of radon and short-lived progeny (Sv)
Ci = daily averaged level (Bq/m3), where the ‘i’ refers to indoors or outdoors
DC = dose coefficient = 9(10−9) (Sv per (Bq h/m3))
EFi = equilibrium factor (unitless, 0.4 for indoors; 0.6 for outdoors)
OFi = occupancy factor (unitless, 0.8 for indoors; 0.2 for outdoors)
i) Living in an area with no fracking wells (background)
Li indicated that particle-bound progeny of Radon-222 contribute to the majority of ambient particle radioactivity measurements that their analysis is based on. We considered four types of sources for this scenario – background indoor and outdoor radon & short-lived progeny, and background indoor and outdoor particle radioactivity.
The population−averaged radon concentration in United States is estimated to be 46 Bq/m3 indoors2,3 and 15 Bq/m3 outdoors4. Li indicated the national average particle radioactivity level (taken as particle radioactivity background) was 0.35 mBq/m3 (0.00035 Bq/m3). Since particle radioactivity originates outdoors, we assumed Cin = Cout for this this source for simplicity.
Using Eqn 1, the following annual effective doses in mSv were estimated for this scenario:
– indoor background radon & short-lived progeny = 1.16
– outdoor background radon & short-lived progeny = 0.14
– indoor background particle radioactivity = <0.0001
– outdoor background particle radioactivity = <0.0001
Total annual effective dose = 1.30 Sv
ii) Living within 20 km of a typical cluster of fracking wells (current)
Li indicated the average particle radioactivity level at oil and gas sites was 0.39 mBq/m3 (0.00039 Bq/m3). This was taken as the particle radioactivity level for this scenario. We considered four types of sources here – background indoor and outdoor radon & short-lived progeny, and indoor and outdoor particle radioactivity within 20 km of a typical cluster of fracking wells. Again, we assumed Cin = Cout for particle radioactivity.
Using Eqn 1, the following annual effective doses in mSV were estimated for this scenario:
– indoor background radon & short-lived progeny = 1.16
– outdoor background outdoors radon & short-lived progeny = 0.14
– indoor particle radioactivity = <0.0001
– outdoor particle radioactivity = <0.0001
Total annual effective dose = 1.30 Sv
iii) Living within 20 km of a typical cluster of fracking wells with an additional 100 wells (future)
Li predicted 100 additional upwind fracking wells within 20 km was associated with an increase of 0.024 mBq/m3 (0.000024 Bq/m3) particle radioactivity downwind. We considered four types of sources here – background indoor and outdoor radon & short-lived progeny, and indoor and outdoor particle radioactivity within 20 km of a typical cluster of fracking wells with an additional 100 wells. Again, we assumed Cin = Cout for particle radioactivity, where Cin = Cout = 0.00039 + 0.000024 = 0.000414 Bq/m3.
Using Eqn 1, the following annual effective doses in mSv were estimated for this scenario:
– indoor background radon & short-lived progeny = 1.16
– outdoor background outdoors radon & short-lived progeny = 0.14
– indoor particle radioactivity = <0.0001
– outdoor particle radioactivity = <0.0001
Total annual effective dose = 1.30 Sv
Appendix references
1 United Nations Scientific Committee on the Effects of Atomic Radiation (UNSCEAR). 2016. Sources, Effects and Risk of Ionizing Radiation, Annex A, Methodology for Estimating Public Exposures due to Radioactive Discharges. https://www.unscear.org/unscear/en/publications/2016.html
2 Nero AV, Schwehr M, Nazaroff W, Revzan K. 1986. Distribution of airborne radon-222 concentrations in U.S. homes. Science, 134, 992−997. https://escholarship.org/content/qt1kj509qt/qt1kj509qt.pdf
3 Marcinowski F, Lucas RM, Yeager WM. 1994. National and regional distributions of airborne radon concentrations in U.S. homes. Health Physics, 66, 699−706. https://www.osti.gov/biblio/229710
4 Hopper RD, Levy RA, Rankin RC, Boyd MA. 1991. National ambient radon study. In: Proceedings of the 1991 International Symposium on Radon and Radon Reduction Technology, 2-5 April 1991, Philadelphia, PA. US Environmental Protection Agency, Research Triangle Park, NC.
via Watts Up With That?
November 2, 2020 at 08:19AM
You must be logged in to post a comment.