Search

Social scientists seeking to inform policy or public action must carefully consider how to identify effects and express inferences because actions based on invalid inferences will not yield the intended results. Recognizing the complexities and uncertainties of social science, we seek to inform inevitable debates about causal inferences by quantifying the conditions necessary to change an inference. Specifically, we review existing sensitivity analyses within the omitted variables and potential outcomes frameworks. We then present the Impact Threshold for a Confounding Variable (ITCV) based on omitted variables in the linear model and the Robustness of Inference to Replacement (RIR) based on the potential outcomes framework. We extend each approach to include benchmarks and to fully account for sampling variability represented by standard errors as well as bias. We exhort social scientists wishing to inform policy and practice to quantify the robustness of their inferences after utilizing the best available data and methods to draw an initial causal inference.

Empirical studies often demonstrate multiple causal mechanisms potentially involving simultaneous or causally related mediators. However, researchers often use simple mediation models to understand the processes because they do not or cannot measure other theoretically relevant mediators. In such cases, another potentially relevant but unobserved mediator potentially confounds the observed mediator, thereby biasing the estimated direct and indirect effects associated with the observed mediator and threatening corresponding inferences. Additionally, researchers may not know the extent to which their measures are reliable, and accordingly, measurement error may bias estimated effects and mislead statistical inferences. Given these threats, we explore how the omission of an unobserved mediator and/or using variables with measurement error biases estimates and affects inferences associated with the observed mediator. Then, building off Frank's impact threshold for a confounding variable (ITCV), we propose a correlation-based sensitivity analysis. Lastly, we provide an R package ConMed to assess the robustness of mediation inferences given the omission of an unobserved, confounding mediator and/or measurement error.

This article reports on quantitative findings from a design-based implementation research (DBIR) project "Carbon TIME" that focused on teaching carbon cycling at multiple scales, from atomic-molecular to global. We report results from a large dataset from diverse schools, including 59,654 student assessments of three-dimensional learning aligned with Next Generation Science Standards collected in classrooms of 133 middle- and high-school teachers over a 4-year period. We used a subset of these data to investigate factors related to students' learning in this large and diverse set of classrooms. Our findings included: First, "Carbon TIME's" curriculum and professional development were associated with substantial improvements in overall student achievement; within classrooms students with lower pretest scores had higher learning gains. Second, differences among individual teachers accounted for more of the variance in student learning gains than other measures including school demographics and students' prior knowledge. Third, school demographic factors (percent free and reduced lunch and percent marginalized students of color) had statistically significant but small effects on students' learning. Finally, sustained investment and participation were important; student learning gains improved across years for the project as a whole, and for individual teachers as they gained experience and participated in professional development. These findings contribute to programs of systemic changes to support three-dimensional science learning.

Carbon cycle pool-and-flux reasoning is a critical facet of climate literacy. This article begins with discussion of why this type of reasoning is both challenging and important. Results from two studies are reported. The first describes students' approaches to carbon cycle pool-and-flux reasoning. The second describes and reports results from an instructional intervention designed to scaffold secondary students' model-based pool-and-flux reasoning. Before instruction, most secondary students employed informal reasoning approaches including good versus bad and correlation heuristics to carbon cycle pool-and-flux problems. After instruction, the portion of students employing goal model-based pool-and-flux reasoning increased from 27 to 52 percent. This study builds on previous and current research to offer a promising instructional approach to scaffolding improvements in students' model-based pool-and-flux reasoning.

Access to treatment and medication for opioid use disorder (MOUD) is essential in reducing opioid use and associated behavioral risks, such as syringe sharing among persons who inject drugs (PWID). Syringe sharing among PWID carries high risk of transmission of serious infections such as hepatitis C and HIV. MOUD resources, such as methadone provider clinics, however, are often unavailable to PWID due to barriers like long travel distance to the nearest methadone provider and the required frequency of clinic visits. The goal of this study is to examine the uncertainty in the effects of travel distance in initiating and continuing methadone treatment and how these interact with different spatial distributions of methadone providers to impact co-injection (syringe sharing) risks. A baseline scenario of spatial access was established using the existing locations of methadone providers in a geographical area of metropolitan Chicago, Illinois, USA. Next, different counterfactual scenarios redistributed the locations of methadone providers in this geographic area according to the densities of both the general adult population and according to the PWID population per zip code. We define different reasonable methadone access assumptions as the combinations of short, medium, and long travel distance preferences combined with three urban/suburban travel distance preference. Our modeling results show that when there is a low travel distance preference for accessing methadone providers, distributing providers near areas that have the greatest need (defined by density of PWID) is best at reducing syringe sharing behaviors. However, this strategy also decreases access across suburban locales, posing even greater difficulty in regions with fewer transit options and providers. As such, without an adequate number of providers to give equitable coverage across the region, spatial distribution cannot be optimized to provide equitable access to all PWID. Our study has important implications for increasing interest in methadone as a resurgent treatment for MOUD in the United States and for guiding policy toward improving access to MOUD among PWID. Author summary: Persons who inject drugs (PWID) such as heroin or other illicit substances are at increased risk of getting or transmitting HIV or hepatitis C (HCV) infection, because limited access to clean syringes and drug injection equipment means that PWID often share contaminated syringes. Public health interventions aimed at reducing injection drug use include medications for opioid use disorder (MOUD) that help to reduce cravings for opioids like heroin and fentanyl. However, resources like MOUD provider clinics are often unavailable to PWID due to factors such as the travel distance from home to clinic, frequent clinic visits, along with social phenomena such as degree of medical mistrust and stigma towards treatment. We use a computational simulation model of PWID social networks and drug use behaviors in Chicago, IL and suburbs to show how the geographic distribution of clinical providers of methadone, an MOUD, impacts the injection frequency of PWID. Our modeling results show that when PWID prefer a low travel distance to methadone providers, distributing providers near areas that have the greatest need is best at reducing syringe sharing behaviors. However, this strategy also decreases access across suburban locales, posing even greater difficulty in regions with fewer transit options and providers. [ABSTRACT FROM AUTHOR]

This article reports on a design-based implementation research (DBIR) project that addresses the question: "How can classrooms be supported at scale to achieve the three-dimensional learning goals of the Next Generation Science Standards?" Inherent in this question are three key design challenges: (i) "three-dimensional learning"--the multidimensional changes in curriculum, assessment, and instruction required for three-dimensional learning; (ii) "scale"--the necessity of change at multiple scales in educational systems; and (iii) "diversity"--achieving rigor in our expectations with responsiveness to the enduring diversity of our students, classrooms, and schools. We discuss findings from the "Carbon TIME" project, which focuses on teaching carbon cycling and energy transformations at multiple scales. Findings focus on design and knowledge building in three interconnected contexts. (i) "Assessment"--understanding and assessing students' three-dimensional learning. Learning progression frameworks provide insight into students' reasoning and the basis for efficient and reliable classroom and large-scale assessments that have used automated scoring of constructed responses for over 80,000 tests. (ii) "Classrooms"--classroom discourse and learning communities. Six "Carbon TIME" units are based on an instructional model that scaffolds students' engagement with phenomena as questioners, investigators, and explainers. The units support substantial learning and reduce the achievement gap between high-pretest and low-pretest students, but with substantial differences among teachers. (iii) "Professional communities"--a professional development course of study and research-practice partnerships address issues of organizational resources, conflicting norms and obligations, and building practical knowledge in schools and districts. Project results show continuing advantages for schools with more organizational resources. Overall, results provide evidence that it is possible to measure and achieve three-dimensional learning at scale. However, this accomplishment requires substantial investments in the material, human, and social resources of educational communities of practice.

COVID-19 surveillance across the United States is essential to tracking and mitigating the pandemic, but data representing cases and deaths may be impacted by attribute, spatial, and temporal uncertainties. COVID-19 case and death data are essential to understanding the pandemic and serve as key inputs for prediction models that inform policy-decisions; consistent information across datasets is critical to ensuring coherent findings. We implement an exploratory data analytic approach to characterize, synthesize, and visualize spatial-temporal dimensions of uncertainty across commonly used datasets for case and death metrics (Johns Hopkins University, the New York Times, USAFacts, and 1Point3Acres). We scrutinize data consistency to assess where and when disagreements occur, potentially indicating underlying uncertainty. We observe differences in cumulative case and death rates to highlight discrepancies and identify spatial patterns. Data are assessed using pairwise agreement (Cohen's kappa) and agreement across all datasets (Fleiss' kappa) to summarize changes over time. Findings suggest highest agreements between CDC, JHU, and NYT datasets. We find nine discrete type-components of information uncertainty for COVID-19 datasets reflecting various complex processes. Understanding processes and indicators of uncertainty in COVID-19 data reporting is especially relevant to public health professionals and policymakers to accurately understand and communicate information about the pandemic. [ABSTRACT FROM AUTHOR]

This article reports on analyses of the instructional practices of six middle‐ and high‐school science teachers in the United States who participated in a research‐practice partnership that aims to support reform science education goals at scale. All six teachers were well qualified, experienced, and locally successful—respected by students, parents, colleagues, and administrators—but they differed in their success in supporting students' three‐dimensional learning. Our goal is to understand how the teachers' instructional practices contributed to their similarities in achieving local success and to differences in enabling students' learning, and to consider the implications of these findings for research‐practice partnerships. Data sources included classroom videos supplemented by interviews with teachers and focus students and examples of student work. We also compared students' learning gains by teacher using pre–post assessments that elicited three‐dimensional performances. Analyses of classroom videos showed how all six teachers achieved local success—they led effectively managed classrooms, covered the curriculum by teaching almost all unit activities, and assessed students' work in fair and efficient ways. There were important differences, however, in how teachers engaged students in science practices. Teachers in classrooms where students achieved lower learning gains followed a pattern of practice we describe as activity‐based teaching, in which students completed investigations and hands‐on activities with few opportunities for sensemaking discussions or three‐dimensional science performances. Teachers whose students achieved higher learning gains combined the social stability characteristic of local classroom success with more demanding instructional practices associated with scientific sensemaking and cognitive apprenticeship. We conclude with a discussion of implications for research‐practice partnerships, highlighting how partnerships need to support all teachers in achieving both local and standards‐based success. [ABSTRACT FROM AUTHOR]