Search

Exposure-response associations between fine particulate matter (PM2.5) and mortality have been extensively studied but potential confounding by daily minimum and maximum temperatures in the weeks preceding death has not been carefully investigated. This paper seeks to close that gap by using lagged partial dependence plots (PDPs), sorted by importance, to quantify how mortality risk depends on lagged values of PM2.5, daily minimum and maximum temperatures and other variables in a dataset from the Los Angeles air basin (SCAQMD). We find that daily minimum and maximum temperatures and daily mortality counts two to three weeks ago are important independent predictors of both current daily elderly mortality and current PM2.5 levels. Thus, it is important to control for these variables over a period of at least several weeks preceding death. Such detailed control for lagged confounders has not been performed in influential past papers on PM2.5-mortality associations, but appears to be essential for isolating the potential causal contributions of specific variables to mortality risk, and, therefore, a worthwhile area for future research and risk assessment modeling.

The use of unverified models for risk estimates and policy recommendations can be highly misleading, as their predictions may not reflect real-world health impacts. For example, a recent article states that NO2 from gas stoves “likely causes ∼50,000 cases of current pediatric asthma from long-term NO2 exposure alone” annually in the United States. This explicitly causal claim, which is contrary to several methodology and review articles published in this journal, among others, reflects both (a) An unverified modeling assumption that pediatric asthma burden is approximately proportional to NO2; and (b) An unverified causal assumption that the assumed proportionality between exposure and response is causal. The article is devoid of any causal analysis showing that these assumptions are likely to be true. It does not show that reducing NO2 exposure from gas stoves would reduce pediatric asthma risk. Its key references report no significant associations – let alone causation – between NO2 and pediatric asthma. Thus, the underlying data suggests that the number of pediatric asthma cases caused by gas stoves in the United States is indistinguishable from zero. This highlights the need to rigorously validate modeling assumptions and causal claims in public health risk assessments to ensure scientifically sound foundations for policy decisions.

Drawing sound causal inferences from observational data is often challenging for both authors and reviewers. This paper discusses the design and application of an Artificial Intelligence Causal Research Assistant (AIA) that seeks to help authors improve causal inferences and conclusions drawn from epidemiological data in health risk assessments. The AIA-assisted review process provides structured reviews and recommendations for improving the causal reasoning, analyses and interpretations made in scientific papers based on epidemiological data. Causal analysis methodologies range from earlier Bradford-Hill considerations to current causal directed acyclic graph (DAG) and related models. AIA seeks to make these methods more accessible and useful to researchers. AIA uses an external script (a “Causal AI Booster” (CAB) program based on classical AI concepts of slot-filling in frames organized into task hierarchies to complete goals) to guide Large Language Models (LLMs), such as OpenAI's ChatGPT or Google's LaMDA (Bard), to systematically review manuscripts and create both (a) recommendations for what to do to improve analyses and reporting; and (b) explanations and support for the recommendations. Review tables and summaries are completed systematically by the LLM in order. For example, recommendations for how to state and caveat causal conclusions in the Abstract and Discussion sections reflect previous analyses of the Study Design and Data Analysis sections. This work illustrates how current AI can contribute to reviewing and providing constructive feedback on research documents. We believe that such AI-assisted review shows promise for enhancing the quality of causal reasoning and exposition in epidemiological studies. It suggests the potential for effective human-AI collaboration in scientific authoring and review processes.

Exposure-response curves are among the most widely used tools of quantitative health risk assessment. However, we propose that exactly what they mean is usually left ambiguous, making it impossible to answer such fundamental questions as whether and by how much reducing exposure by a stated amount would change average population risks and distributions of individual risks. Recent concepts and computational methods from causal artificial intelligence (CAI) and machine learning (ML) can be applied to clarify what an exposure-response curve means; what other variables are held fixed (and at what levels) in estimating it; and how much inter-individual variability there is around population average exposure-response curves. These advances in conceptual clarity and practical computational methods not only enable epidemiologists and risk analysis practitioners to better quantify population and individual exposure-response curves but also challenge them to specify exactly what exposure-response relationships they seek to quantify and communicate to risk managers and how to use the resulting information to improve risk management decisions.

Several recent news stories have alarmed many politicians and members of the public by reporting that indoor air pollution from gas stoves causes about 13% of childhood asthma in the United States. Research on the reproducibility and trustworthiness of epidemiological risk assessments has identified a number of common questionable research practices (QRPs) that should be avoided to draw sound causal conclusions from epidemiological data. Examples of such QRPs include claiming causation without using study designs or data analyses that allow valid causal inferences; generalizing or transporting risk estimates based on data for specific populations, time periods, and locations to different ones without accounting for differences in the study and target populations; claiming causation without discussing or quantitatively correcting for confounding, external validity bias, or other biases; and not mentioning or resolving contradictory evidence. We examine the recently estimated gas stove-childhood asthma associations from the perspective of these QRPs and conclude that it exemplifies all of them. The quantitative claim that about 13% of childhood asthma in the United States could be prevented by reducing exposure to gas stove pollution is not supported by the data collected or by the measures of association (Population Attributable Fractions) used to analyze the data. The qualitative finding that reducing exposure to gas stove pollution would reduce the burden of childhood asthma in the United States has no demonstrated validity. Systematically checking how and whether QRPs have been addressed before reporting or responding to claims that everyday exposures cause substantial harm to health might reduce social amplification of perceived risks based on QRPs and help to improve the credibility and trustworthiness of published epidemiological risk assessments.

We present a Socratic dialogue with ChatGPT, a large language model (LLM), on the causal interpretation of epidemiological associations between fine particulate matter (PM2.5) and human mortality risks. ChatGPT, reflecting probable patterns of human reasoning and argumentation in the sources on which it has been trained, initially holds that “It is well-established that exposure to ambient levels of PM2.5 does increase mortality risk” and adds the unsolicited remark that “Reducing exposure to PM2.5 is an important public health priority.” After patient questioning, however, it concludes that “It is not known with certainty that current ambient levels of PM2.5 increase mortality risk. While there is strong evidence of an association between PM2.5 and mortality risk, the causal nature of this association remains uncertain due to the possibility of omitted confounders.” This revised evaluation of the evidence suggests the potential value of sustained questioning in refining and improving both the types of human reasoning and argumentation imitated by current LLMs and the reliability of the initial conclusions expressed by current LLMs.

Do faster slaughter line speeds for young chickens increase risk of Salmonella contamination? We analyze data collected in 2018–2019 from 97 slaughter establishments processing young chickens to examine the extent to which differences in slaughter line speeds across establishments operating under the same inspection system explain observed differences in their microbial quality, specifically frequencies of positive Salmonella samples. A variety of off-the-shelf statistical and machine learning techniques applied to the data to identify and visualize correlations and potential causal relationships among variables showed that the presence of Salmonella or other indicators of process control, such as noncompliance records for regulations associated with process control and food safety, are not significantly increased in establishments with higher line speeds (e.g., above 140 birds per min) compared with establishments with lower line speeds when establishments are operating under the conditions present in this study. This included some establishments operating under specific criteria to obtain a waiver for line speed. A null hypothesis advanced over 30 yr ago by the National Research Council that increased line speeds result in a product that is not contaminated more often than before line speeds were increased, appears to be fully consistent with these recent data.

Causal inference regarding exposures to ambient fine particulate matter (PM2.5) and mortality estimated from observational studies is limited by confounding, among other factors. In light of a variety of causal inference frameworks and methods that have been developed over the past century to specifically quantify causal effects, three research teams were selected in 2016 to evaluate the causality of PM2.5-mortality association among Medicare beneficiaries, using their own selections of causal inference methods and study designs but the same data sources. With a particular focus on controlling for unmeasured confounding, two research teams adopted an instrumental variables approach under a quasi-experiment or natural experiment study design, whereas one team adopted a structural nested mean model under the traditional cohort study design. All three research teams reported results supporting an estimated counterfactual causal relationship between ambient PM2.5 and all-cause mortality, and their estimated causal relationships are largely of similar magnitudes to recent epidemiological studies based on regression analyses with omitted potential confounders. The causal methods used by all three research teams were built upon the potential outcomes framework. This framework has marked conceptual advantages over regression-based methods in addressing confounding and yielding unbiased estimates of average treatment effect in observational epidemiological studies. However, potential violations of the unverifiable assumptions underlying each causal method leave the results from all three studies subject to biases. We also note that the studies are not immune to some other common sources of bias, including exposure measurement errors, ecological study design, model uncertainty and specification errors, and irrelevant exposure windows, that can undermine the validity of causal inferences in observational studies. As a result, despite some apparent consistency of study results from the three research teams with the wider epidemiological literature on PM2.5-mortality statistical associations, caution seems warranted in drawing causal conclusions from the results. A possible way forward is to improve study design and reduce dependence of conclusions on untested assumptions by complementing potential outcomes methods with structural causal modeling and information-theoretic methods that emphasize empirically tested and validated relationships.

We argue that population attributable fractions, probabilities of causation, burdens of disease, and similar association-based measures often do not provide valid estimates or surrogates for the fraction or number of disease cases that would be prevented by eliminating or reducing an exposure because their calculations do not include crucial mechanistic information. We use a thought experiment with a cascade of dominos to illustrate the need for mechanistic information when answering questions about how changing exposures changes risk. We suggest that modern methods of causal artificial intelligence (CAI) can fill this gap: they can complement and extend traditional epidemiological attribution calculations to provide information useful for risk management decisions.

Population attributable fraction (PAF), probability of causation, burden of disease, and related quantities derived from relative risk ratios are widely used in applied epidemiology and health risk analysis to quantify the extent to which reducing or eliminating exposures would reduce disease risks. This causal interpretation conflates association with causation. It has sometimes led to demonstrably mistaken predictions and ineffective risk management recommendations. Causal artificial intelligence (CAI) methods developed at the intersection of many scientific disciplines over the past century instead use quantitative high-level descriptions of networks of causal mechanisms (typically represented by conditional probability tables or structural equations) to predict the effects caused by interventions. We summarize these developments and discuss how CAI methods can be applied to realistically imperfect data and knowledge – e.g., with unobserved (latent) variables, missing data, measurement errors, interindividual heterogeneity in exposure-response functions, and model uncertainty. We recommend that CAI methods can help to improve the conceptual foundations and practical value of epidemiological calculations by replacing association-based attributions of risk to exposures or other risk factors with causal predictions of the changes in health effects caused by interventions.

In the first half of 2020, much excitement in news media and some peer reviewed scientific articles was generated by the discovery that fine particulate matter (PM2.5) concentrations and COVID-19 mortality rates are statistically significantly positively associated in some regression models. This article points out that they are non-significantly negatively associated in other regression models, once omitted confounders (such as latitude and longitude) are included. More importantly, positive regression coefficients can and do arise when (generalized) linear regression models are applied to data with strong nonlinearities, including data on PM2.5, population density, and COVID-19 mortality rates, due to model specification errors. In general, statistical modeling accompanied by judgments about causal interpretations of statistical associations and regression coefficients – the current weight-of-evidence (WoE) approach favored in much current regulatory risk analysis for air pollutants – is not a valid basis for determining whether or to what extent risk of harm to human health would be reduced by reducing exposure. The traditional scientific method based on testing predictive generalizations against data remains a more reliable paradigm for risk analysis and risk management.

A proposed matrix bridging between the results supplied by epidemiologists and those demanded by risk assessors proposes that a key piece of information sought by risk assessors is the shape of the exposure-response curve (e.g., linear vs. nonlinear, threshold vs. no threshold, etc.). This comment emphasizes that there are several different exposure-response curves, having different causal interpretations and risk management implications. Risk assessors and risk management decision makers and policy maker usually need to know how changes in exposures would change disease risks (given assumptions about levels of other variables). Epidemiologists typically provide conditional expected observed values of response variables for different observed levels of exposures, i.e., regression relationships. These are two different curves and may have quite different shapes. Current widespread epidemiological practice conflates them. Being clear about what is needed for risk assessment (usually the former) and what has been produced by epidemiologists (usually the latter) can help to identify mismatches and to build more useful bridges between these communities.

In most areas of applied research, sound science entails use of clear definitions, explicit derivations of conclusions, reproducible tests of predictions against observations, and careful qualification of causal interpretations and conclusions to acknowledge remaining ambiguities or conflicts in evidence. I propose that these same principles should also be applied to the study of air pollution health effects associated with exposures to air pollution. This contrasts with a popular weight-of-evidence approach that does not require these restrictions, but that instead draws on authoritative judgments by selected experts to reach policy-relevant conclusions. Although the scientific approach summarized here may be less popular and more exacting, I propose that it is essential for reaching valid conclusions and identifying causally effective policies to protect public health, and that it should therefore be adopted in regulatory risk analysis.

How can scientists and risk assessors best communicate with each other, the media, the public, and policy makers what is known, what is guessed, and what is still unknown or uncertain about how changes in air pollution affect human mortality? Current practice includes emphatic pronouncements, striking headlines, and colorful infographics about deaths attributed to air pollution. Important qualifications, uncertainties, and unverified assumptions are often buried deeply in technical paper if they are stated at all. Although some sensational claims about air pollution mortalities have proved to be mistaken, errors and corrections receive little or no attention from media, policy makers, or interest groups who helped to spread the original sensational claims. More encouragingly, substantial recent progress has been made in technical methods for quantifying and communicating what is known about how reducing exposure affects health risks and uncertainties. These advances make it possible to better describe and explain both what we now know and what we are still uncertain about for human health effects of interventions that reduce pollution levels. We review progress in accurate communication about deaths caused by air pollution and discuss how advances in quantitative risk assessment and risk communication can provide a more useful basis for informing public opinion and policy deliberations – one that is more accurate and that answers more relevant questions – than the infographics and sensational claims that dominate risk communication about air pollution deaths today.

Holistic expert judgments about causality are widely used in regulatory risk assessments, with causal determination categories being used to summarize huge amounts of complex evidence and to help inform and drive major regulatory decisions. The causal determination categories used typically cover a relatively narrow range (e.g., from “causal relationship,” “likely to be a causal relationship,” or “suggestive of a causal relationship” to “inadequate to infer a causal relationship” and “not likely to be a causal relationship”). Other categories, such as “not a causal relationship” or “likely to not be a causal relationship,” are omitted entirely. We note fundamental limitations of such procedures. A few categories cannot encode most of the wide variations in evidence about risks and causal exposure-response relationships found in both theory and practice. “Causal relationship” is usually left undefined, and may be interpreted very differently by different people. Whether it refers to direct, indirect, or total causal effects is seldom specified. We propose that causal partial dependence plots of predicted risk against exposure, calculated from conditional probability tables (CPTs) or models that satisfy an empirically testable condition of invariant causal prediction (ICP) across studies, can provide much more useful and clearly defined information to decision-makers. This alternative framework treats causal relationships, and evidence about them, as continuous and quantitative rather than categorical and qualitative. This is not only advantageous for clarity and realism, but it encourages better use of data and scientific method, including applying independently verifiable tests to inform conclusions about how and whether changes in exposures would change individual and population health risks.