Search

Introduction: Hip fractures among older people are a major public health issue, which can impact quality of life and increase mortality within the year after they occur. A recent observational study found an increased risk of hip fracture in subjects who were new users of tramadol compared with codeine. These drugs have somewhat different indications. Tramadol is indicated for moderate to severe pain and can be used for an extended period; codeine is indicated for mild to moderate pain and cough suppression. Objective: In this observational study, we compared the risk of hip fracture in new users of tramadol or codeine, using multiple databases and analytical methods. Methods: Using data from the Clinical Practice Research Datalink and three US claims databases, we compared the risk of hip fracture after exposure to tramadol or codeine in subjects aged 50–89 years. To ensure comparability, large-scale propensity scores were used to adjust for confounding. Results: We observed a calibrated hazard ratio of 1.10 (95% calibrated confidence interval 0.99–1.21) in the Clinical Practice Research Datalink database, and a pooled estimate across the US databases yielded a calibrated hazard ratio of 1.06 (95% calibrated confidence interval 0.97–1.16). Conclusions: Our results did not demonstrate a statistically significant difference between subjects treated for pain with tramadol compared with codeine for the outcome of hip fracture risk.

Objective To develop a lossless distributed algorithm for generalized linear mixed model (GLMM) with application to privacy-preserving hospital profiling. Materials and Methods The GLMM is often fitted to implement hospital profiling, using clinical or administrative claims data. Due to individual patient data (IPD) privacy regulations and the computational complexity of GLMM, a distributed algorithm for hospital profiling is needed. We develop a novel distributed penalized quasi-likelihood (dPQL) algorithm to fit GLMM when only aggregated data, rather than IPD, can be shared across hospitals. We also show that the standardized mortality rates, which are often reported as the results of hospital profiling, can also be calculated distributively without sharing IPD. We demonstrate the applicability of the proposed dPQL algorithm by ranking 929 hospitals for coronavirus disease 2019 (COVID-19) mortality or referral to hospice that have been previously studied. Results The proposed dPQL algorithm is mathematically proven to be lossless, that is, it obtains identical results as if IPD were pooled from all hospitals. In the example of hospital profiling regarding COVID-19 mortality, the dPQL algorithm reached convergence with only 5 iterations, and the estimation of fixed effects, random effects, and mortality rates were identical to that of the PQL from pooled data. Conclusion The dPQL algorithm is lossless, privacy-preserving and fast-converging for fitting GLMM. It provides an extremely suitable and convenient distributed approach for hospital profiling.

Background: Adverse events of special interest (AESIs) were pre-specified to be monitored for the COVID-19 vaccines. Some AESIs are not only associated with the vaccines, but with COVID-19. Our aim was to characterise the incidence rates of AESIs following SARS-CoV-2 infection in patients and compare these to historical rates in the general population. Methods: A multi-national cohort study with data from primary care, electronic health records, and insurance claims mapped to a common data model. This study's evidence was collected between Jan 1, 2017 and the conclusion of each database (which ranged from Jul 2020 to May 2022). The 16 pre-specified prevalent AESIs were: acute myocardial infarction, anaphylaxis, appendicitis, Bell's palsy, deep vein thrombosis, disseminated intravascular coagulation, encephalomyelitis, Guillain- Barré syndrome, haemorrhagic stroke, non-haemorrhagic stroke, immune thrombocytopenia, myocarditis/pericarditis, narcolepsy, pulmonary embolism, transverse myelitis, and thrombosis with thrombocytopenia. Age-sex standardised incidence rate ratios (SIR) were estimated to compare post-COVID-19 to pre-pandemic rates in each of the databases. Findings: Substantial heterogeneity by age was seen for AESI rates, with some clearly increasing with age but others following the opposite trend. Similarly, differences were also observed across databases for same health outcome and age-sex strata. All studied AESIs appeared consistently more common in the post-COVID-19 compared to the historical cohorts, with related meta-analytic SIRs ranging from 1.32 (1.05 to 1.66) for narcolepsy to 11.70 (10.10 to 13.70) for pulmonary embolism. Interpretation: Our findings suggest all AESIs are more common after COVID-19 than in the general population. Thromboembolic events were particularly common, and over 10-fold more so. More research is needed to contextualise post-COVID-19 complications in the longer term. Funding: None.

ObjectivesTo assess the uptake of second-line antihyperglycemic agents among patients with type-2 diabetes mellitus (T2DM) receiving metformin.DesignSerial cross-sectional study (2011-2021).SettingTen US and seven non-US electronic health record and administrative claims databases in the Observational Health Data Sciences and Informatics network.Participants4.8 million patients with T2DM receiving metformin.Main Outcomes MeasuresCalendar-year trends in the proportional initiation of second-line antihyperglycemic agents, glucagon-like peptide-1 receptor agonists (GLP-1 RAs), sodium-glucose cotransporter 2 inhibitors (SGLT2is), dipeptidyl peptidase-4 inhibitors, and sulfonylureas, for each database. We also evaluated the relative drug class-level uptake across cardiovascular risk groups.ResultsWe identified 4.6 million patients with T2DM in US databases, 61,382 from Spain, 32,442 from Germany, 25,173 from the UK, 13,270 from France, 5,580 from Scotland, 4,614 from Hong Kong, and 2,322 from Australia. During 2011-2021, the combined proportional initiation of cardioprotective antihyperglycemic agents, GLP-1 RAs and SGLT2is, increased across all data sources, with the combined initiation of these drugs as second-line agents in 2021 ranging from 35.2% to 68.2% in the US databases, 15.4% in France, 34.7% in Spain, 50.1% in Germany, and 54.8% in Scotland. From 2016 to 2021, in some US and non-US databases, uptake of GLP-1 RAs and SGLT2is increased more significantly among populations without cardiovascular disease compared to those with established cardiovascular disease, without any data source providing evidence of a greater increase in their uptake in the populations with cardiovascular disease.ConclusionsDespite the increase in overall uptake of cardioprotective antihyperglycemic agents as second-line treatment for T2DM, their uptake was lower in patients with cardiovascular disease over the last decade. A strategy to ensure medication use concordant with guideline recommendations is essential to improve outcomes of patients with T2DM.

Background Prognostic models that are accurate could help aid medical decision making. Large observational databases often contain temporal medical data for large and diverse populations of patients. It may be possible to learn prognostic models using the large observational data. Often the performance of a prognostic model undesirably worsens when transported to a different database (or into a clinical setting). In this study we investigate different ensemble approaches that combine prognostic models independently developed using different databases (a simple federated learning approach) to determine whether ensembles that combine models developed across databases can improve model transportability (perform better in new data than single database models)? Methods For a given prediction question we independently trained five single database models each using a different observational healthcare database. We then developed and investigated numerous ensemble models (fusion, stacking and mixture of experts) that combined the different database models. Performance of each model was investigated via discrimination and calibration using a leave one dataset out technique, i.e., hold out one database to use for validation and use the remaining four datasets for model development. The internal validation of a model developed using the hold out database was calculated and presented as the ‘internal benchmark’ for comparison. Results In this study the fusion ensembles generally outperformed the single database models when transported to a previously unseen database and the performances were more consistent across unseen databases. Stacking ensembles performed poorly in terms of discrimination when the labels in the unseen database were limited. Calibration was consistently poor when both ensembles and single database models were applied to previously unseen databases. Conclusion A simple federated learning approach that implements ensemble techniques to combine models independently developed across different databases for the same prediction question may improve the discriminative performance in new data (new database or clinical setting) but will need to be recalibrated using the new data. This could help medical decision making by improving prognostic model performance.

Purpose: Alpha-1 blockers, often used to treat benign prostatic hyperplasia (BPH), have been hypothesized to prevent COVID-19 complications by minimising cytokine storm release. The proposed treatment based on this hypothesis currently lacks support from reliable real-world evidence, however. We leverage an international network of large-scale healthcare databases to generate comprehensive evidence in a transparent and reproducible manner.Methods: In this international cohort study, we deployed electronic health records from Spain (SIDIAP) and the United States (Department of Veterans Affairs, Columbia University Irving Medical Center, IQVIA OpenClaims, Optum DOD, Optum EHR). We assessed association between alpha-1 blocker use and risks of three COVID-19 outcomes—diagnosis, hospitalization, and hospitalization requiring intensive services—using a prevalent-user active-comparator design. We estimated hazard ratios using state-of-the-art techniques to minimize potential confounding, including large-scale propensity score matching/stratification and negative control calibration. We pooled database-specific estimates through random effects meta-analysis.Results: Our study overall included 2.6 and 0.46 million users of alpha-1 blockers and of alternative BPH medications. We observed no significant difference in their risks for any of the COVID-19 outcomes, with our meta-analytic HR estimates being 1.02 (95% CI: 0.92–1.13) for diagnosis, 1.00 (95% CI: 0.89–1.13) for hospitalization, and 1.15 (95% CI: 0.71–1.88) for hospitalization requiring intensive services.Conclusion: We found no evidence of the hypothesized reduction in risks of the COVID-19 outcomes from the prevalent-use of alpha-1 blockers—further research is needed to identify effective therapies for this novel disease.

Post-approval safety surveillance of medical products using observational healthcare data can help identify safety issues beyond those found in pre-approval trials. When testing sequentially as data accrue, maximum sequential probability ratio testing (MaxSPRT) is a common approach to maintaining nominal type 1 error. However, the true type 1 error may still deviate from the specified one because of systematic error due to the observational nature of the analysis. This systematic error may persist even after controlling for known confounders. Here we propose to address this issue by combing MaxSPRT with empirical calibration. In empirical calibration, we assume uncertainty about the systematic error in our analysis, the source of uncertainty commonly overlooked in practice. We infer a probability distribution of systematic error by relying on a large set of negative controls: exposure-outcome where no causal effect is believed to exist. Integrating this distribution into our test statistics has previously been shown to restore type 1 error to nominal. Here we show how we can calibrate the critical value central to MaxSPRT. We evaluate this novel approach using simulations and real electronic health records, using H1N1 vaccinations during the 2009-2010 season as an example. Results show that combining empirical calibration with MaxSPRT restores nominal type 1 error. In our real-world example, adjusting for systematic error using empirical calibration has a larger impact than, and hence is just as essential as, adjusting for sequential testing using MaxSPRT. We recommend performing both, using the method described here., Supplemental Materials are available at https://github.com/ohdsi-studies/Eumaeus/tree/main/extras/EmpiricalCalibrationMaxSprtSuppl

Evidence for the effectiveness and safety of the third-generation beta-blockers other than atenolol in hypertension remains scarce. We assessed the effectiveness and safety of beta-blockers as first-line treatment for hypertension using three databases in the United States: two administrative claim databases and one electronic health record-based database from 2001 to 2018. In each database, comparative effectiveness of beta-blockers for the risks of acute myocardial infarction, stroke, and hospitalization for heart failure was assessed, using large-scale propensity adjustment and empirical calibration. Estimates were combined across databases using random-effects meta-analyses. Overall, 118,133 and 267,891 patients initiated third-generation beta-blockers (carvedilol and nebivolol) or atenolol, respectively. The pooled hazard ratios of acute myocardial infarction, stroke, hospitalization for heart failure, and most metabolic complications were not different between the third-generation beta-blockers versus atenolol after propensity score matching and empirical calibration (hazard ratio 1.07, 95% CI 0.74 to 1.55 for acute myocardial infarction; hazard ratio 1.06, 95% CI 0.87 to 1.31 for stroke; hazard ratio 1.46, 95% CI 0.99 to 2.24 for hospitalized heart failure). Third-generation beta-blockers were associated with significantly higher risk of stroke than angiotensin-converting enzyme inhibitors (hazard ratio 1.29, 95% CI 1.03 to 1.72), and thiazide diuretics (hazard ratio 1.56, 95% CI 1.17 to 2.20). In conclusion, this study found many patients with first-line beta-blocker monotherapy for hypertension and no statistically significant differences in the effectiveness and safety comparing atenolol with third-generation beta-blockers. Patients on third-generation beta-blockers had a higher risk of stroke than those on angiotensin-converting enzyme inhibitors and thiazide diuretics.

Background Angiotensin-converting enzyme inhibitors (ACEIs) and angiotensin receptor blockers (ARBs) have been postulated to affect susceptibility to COVID-19. Observational studies so far have lacked rigorous ascertainment adjustment and international generalisability. We aimed to determine whether use of ACEIs or ARBs is associated with an increased susceptibility to COVID-19 in patients with hypertension. Methods In this international, open science, cohort analysis, we used electronic health records from Spain (Information Systems for Research in Primary Care [SIDIAP]) and the USA (Columbia University Irving Medical Center data warehouse [CUIMC] and Department of Veterans Affairs Observational Medical Outcomes Partnership [VA-OMOP]) to identify patients aged 18 years or older with at least one prescription for ACEIs and ARBs (target cohort) or calcium channel blockers (CCBs) and thiazide or thiazide-like diuretics (THZs; comparator cohort) between Nov 1, 2019, and Jan 31, 2020. Users were defined separately as receiving either monotherapy with these four drug classes, or monotherapy or combination therapy (combination use) with other antihypertensive medications. We assessed four outcomes: COVID-19 diagnosis; hospital admission with COVID-19; hospital admission with pneumonia; and hospital admission with pneumonia, acute respiratory distress syndrome, acute kidney injury, or sepsis. We built large-scale propensity score methods derived through a data-driven approach and negative control experiments across ten pairwise comparisons, with results meta-analysed to generate 1280 study effects. For each study effect, we did negative control outcome experiments using a possible 123 controls identified through a data-rich algorithm. This process used a set of predefined baseline patient characteristics to provide the most accurate prediction of treatment and balance among patient cohorts across characteristics. The study is registered with the EU Post-Authorisation Studies register, EUPAS35296. Findings Among 1 355 349 antihypertensive users (363 785 ACEI or ARB monotherapy users, 248 915 CCB or THZ monotherapy users, 711 799 ACEI or ARB combination users, and 473 076 CCB or THZ combination users) included in analyses, no association was observed between COVID-19 diagnosis and exposure to ACEI or ARB monotherapy versus CCB or THZ monotherapy (calibrated hazard ratio [HR] 0·98, 95% CI 0·84–1·14) or combination use exposure (1·01, 0·90–1·15). ACEIs alone similarly showed no relative risk difference when compared with CCB or THZ monotherapy (HR 0·91, 95% CI 0·68–1·21; with heterogeneity of >40%) or combination use (0·95, 0·83–1·07). Directly comparing ACEIs with ARBs demonstrated a moderately lower risk with ACEIs, which was significant with combination use (HR 0·88, 95% CI 0·79–0·99) and non-significant for monotherapy (0·85, 0·69–1·05). We observed no significant difference between drug classes for risk of hospital admission with COVID-19, hospital admission with pneumonia, or hospital admission with pneumonia, acute respiratory distress syndrome, acute kidney injury, or sepsis across all comparisons. Interpretation No clinically significant increased risk of COVID-19 diagnosis or hospital admission-related outcomes associated with ACEI or ARB use was observed, suggesting users should not discontinue or change their treatment to decrease their risk of COVID-19.

The use of antipsychotic medications (APMs) could be different among countries due to availability, approved indications, characteristics and clinical practice. However, there is limited literature providing comparisons of APMs use among countries. To examine trends in antipsychotic prescribing in Taiwan, Hong Kong, Japan, and the United States, we conducted a cross-national study from 2002 to 2014 b y using the distributed network approach with common data model. We included all patients who had at least a record of antipsychotic prescription in this study, and defined patients without previous exposure of antipsychotics for 6 months before the index date as new users for incidence estimation. We calculated the incidence, prevalence, and prescription rate of each medication by calendar year. Among older patients, sulpiride was the most incident [incidence rate (IR) 11.0–23.3) and prevalent [prevalence rate (PR) 11.9–14.3) APM in Taiwan, and most prevalent (PR 2.5–3.9) in Japan. Quetiapine and haloperidol were most common in the United States (IR 8.1–9.5; PR 18.0–18.4) and Hong Kong (PR 8.8–13.7; PR 10.6–12.7), respectively. The trend of quetiapine use was increasing in Taiwan, Hong Kong and the United States. As compared to older patients, the younger patients had more propensity to be prescribed second-generation APM for treatment in four countries. Trends in antipsychotic prescribing varied among countries. Quetiapine use was most prevalent in the United States and increasing in Taiwan and Hong Kong. The increasing use of quetiapine in the elderly patients might be due to its safety profile compared to other APMs.

Hip fractures among older people are a major public health issue, which can impact quality of life and increase mortality within the year after they occur. A recent observational study found an increased risk of hip fracture in subjects who were new users of tramadol compared with codeine. These drugs have somewhat different indications. Tramadol is indicated for moderate to severe pain and can be used for an extended period; codeine is indicated for mild to moderate pain and cough suppression.In this observational study, we compared the risk of hip fracture in new users of tramadol or codeine, using multiple databases and analytical methods.Using data from the Clinical Practice Research Datalink and three US claims databases, we compared the risk of hip fracture after exposure to tramadol or codeine in subjects aged 50-89 years. To ensure comparability, large-scale propensity scores were used to adjust for confounding.We observed a calibrated hazard ratio of 1.10 (95% calibrated confidence interval 0.99-1.21) in the Clinical Practice Research Datalink database, and a pooled estimate across the US databases yielded a calibrated hazard ratio of 1.06 (95% calibrated confidence interval 0.97-1.16).Our results did not demonstrate a statistically significant difference between subjects treated for pain with tramadol compared with codeine for the outcome of hip fracture risk.

Cardinality matching (CM), a novel matching technique, finds the largest matched sample meeting prespecified balance criteria thereby overcoming limitations of propensity score matching (PSM) associated with limited covariate overlap, which are especially pronounced in studies with small sample sizes. The current study proposes a framework for large-scale CM (LS-CM); and compares large-scale PSM (LS-PSM) and LS-CM in terms of post-match sample size, covariate balance and residual confounding at progressively smaller sample sizes.Evaluation of LS-PSM and LS-CM within a comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and thiazide or thiazide-like diuretic monotherapy identified from a U.S. insurance claims database. Candidate covariates included patient demographics, and all observed prior conditions, drug exposures and procedures. Propensity scores were calculated using LASSO regression, and candidate covariates with non-zero beta coefficients in the propensity model were defined as matching covariates for use in LS-CM. One-to-one matching was performed using progressively tighter parameter settings. Covariate balance was assessed using standardized mean differences. Hazard ratios for negative control outcomes perceived as unassociated with treatment (i.e., true hazard ratio of 1) were estimated using unconditional Cox models. Residual confounding was assessed using the expected systematic error of the empirical null distribution of negative control effect estimates compared to the ground truth. To simulate diverse research conditions, analyses were repeated within 10 %, 1 and 0.5 % subsample groups with increasingly limited covariate overlap.A total of 172,117 patients (ACEI: 129,078; thiazide: 43,039) met the study criteria. As compared to LS-PSM, LS-CM was associated with increased sample retention. Although LS-PSM achieved balance across all matching covariates within the full study population, substantial matching covariate imbalance was observed within the 1 and 0.5 % subsample groups. Meanwhile, LS-CM achieved matching covariate balance across all analyses. LS-PSM was associated with better candidate covariate balance within the full study population. Otherwise, both matching techniques achieved comparable candidate covariate balance and expected systematic error.LS-CM found the largest matched sample meeting prespecified balance criteria while achieving comparable candidate covariate balance and residual confounding. We recommend LS-CM as an alternative to LS-PSM in studies with small sample sizes or limited covariate overlap.

Evidence derived from existing health-care data, such as administrative claims and electronic health records, can fill evidence gaps in medicine. However, many claim such data cannot be used to estimate causal treatment effects because of the potential for observational study bias; for example, due to residual confounding. Other concerns include P hacking and publication bias. In response, the Observational Health Data Sciences and Informatics international collaborative launched the Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND) research initiative. Its mission is to generate evidence on the effects of medical interventions using observational health-care databases while addressing the aforementioned concerns by following a recently proposed paradigm. We define 10 principles of LEGEND that enshrine this new paradigm, prescribing the generation and dissemination of evidence on many research questions at once; for example, comparing all treatments for a disease for many outcomes, thus preventing publication bias. These questions are answered using a prespecified and systematic approach, avoiding P hacking. Best-practice statistical methods address measured confounding, and control questions (research questions where the answer is known) quantify potential residual bias. Finally, the evidence is generated in a network of databases to assess consistency by sharing open-source analytics code to enhance transparency and reproducibility, but without sharing patient-level information. Here we detail the LEGEND principles and provide a generic overview of a LEGEND study. Our companion paper highlights an example study on the effects of hypertension treatments, and evaluates the internal and external validity of the evidence we generate.

Introduction Observational studies estimating severe outcomes for paracetamol versus ibuprofen use have acknowledged the specific challenge of channeling bias. A previous study relying on negative controls suggested that using large-scale propensity score (LSPS) matching may mitigate bias better than models using limited lists of covariates. Objective The aim was to assess whether using LSPS matching would enable the evaluation of paracetamol, compared to ibuprofen, and increased risk of myocardial infarction, stroke, gastrointestinal (GI) bleeding, or acute renal failure. Study design and setting In a new-user cohort study, we used two propensity score model strategies for confounder controls. One replicated the approach of controlling for a hand-picked list. The second used LSPSs based on all available covariates for matching. Positive and negative controls assessed residual confounding and calibrated confidence intervals. The data source was the Clinical Practices Research Datalink (CPRD). Results A substantial proportion of negative controls were statistically significant after propensity score matching on the publication covariates, indicating considerable systematic error. LSPS adjustment was less biased, but residual error remained. The calibrated estimates resulted in very wide confidence intervals, indicating large uncertainty in effect estimates once residual error was incorporated. Conclusions For paracetamol versus ibuprofen, when using LSPS methods in the CPRD, it is only possible to distinguish true effects if those effects are large (hazard ratio > 2). Due to their smaller hazard ratios, the outcomes under study cannot be differentiated from null effects (represented by negative controls) even if there were a true effect. Based on these data, we conclude that we are unable to determine whether paracetamol is associated with an increased risk of myocardial infarction, stroke, GI bleeding, and acute renal failure compared to ibuprofen, due to residual confounding. Electronic supplementary material The online version of this article (10.1007/s40264-020-00950-3) contains supplementary material, which is available to authorized users.

Objective: Observational evidence suggests that patients with type 2 diabetes mellitus (T2DM) are at increased risk for acute pancreatitis (AP) versus those without T2DM. A small number of AP events were reported in clinical trials of the sodium glucose co-transporter 2 inhibitor canagliflozin, though no imbalances were observed between treatment groups. This observational study evaluated risk of AP among new users of canagliflozin compared with new users of six classes of other antihyperglycemic agents (AHAs). Methods: Three US claims databases were analyzed based on a prespecified protocol approved by the European Medicines Agency. Propensity score adjustment controlled for imbalances in baseline covariates. Cox regression models estimated the hazard ratio of AP with canagliflozin compared with other AHAs using on-treatment (primary) and intent-to-treat approaches. Sensitivity analyses assessed robustness of findings. Results: Across the three databases, there were between 12,023–80,986 new users of canagliflozin; the unadjusted incidence rates of AP (per 1000 person-years) were between 1.5–2.2 for canagliflozin and 1.1–6.6 for other AHAs. The risk of AP was generally similar for new users of canagliflozin compared with new users of glucagon-like peptide-1 receptor agonists, dipeptidyl peptidase-4 inhibitors, sulfonylureas, thiazolidinediones, insulin, and other AHAs, with no consistent between-treatment differences observed across databases. Intent-to-treat and sensitivity analysis findings were qualitatively consistent with on-treatment findings. Conclusions: In this large observational study, incidence rates of AP in patients with T2DM treated with canagliflozin or other AHAs were generally similar, with no evidence suggesting that canagliflozin is associated with increased risk of AP compared with other AHAs.

BackgroundVaccine safety surveillance commonly includes a serial testing approach with a sensitive method for “signal generation” and specific method for “signal validation.” Whether serially combining epidemiological designs improves both sensitivity and specificity is unknown.MethodsWe assessed the overall performance of serial testing using three administrative claims and one electronic health record database. We compared Type I and II errors before and after empirical calibration for historical comparator, SCCS, and the serial combination of those designs against six vaccine exposure groups with 93 negative control and 279 imputed positive control outcomes.ResultsHistorical comparator mostly had lower Type II error than SCCS. SCCS had lower Type I error than the historical comparator. Before empirical calibration, serial combination increased specificity and decreased sensitivity. Type II errors mostly exceeded 50%. After empirical calibration, Type I errors returned to nominal; sensitivity was lowest when the methods were combined.ConclusionWe recommend against the serial approach in vaccine safety surveillance. While serial combination produced fewer false positive signals compared to the most specific method, it generated more false negative signals compared to the most sensitive method. Using the noisy historical comparator in front of SCCS deteriorated overall performance in evaluating safety signals.Key MessagesUsing the serial approach in vaccine safety surveillance did not improve overall performance: specificity increased but sensitivity decreased.Without empirical calibration, Type II errors exceeded 50%; after empirical calibration, Type I error rates returned to nominal with negligible change to Type II error rates.While prior research has suggested high sensitivity of the historical comparator method in distinguishing true safety signals, there were cases when self-controlled case series was more sensitive.Vaccine safety surveillance is becoming increasingly important, so monitoring systems should closely consider the utility and sequence of epidemiological designs.

Background: Routinely collected healthcare data such as administrative claims and electronic health records (EHR) can complement clinical trials and spontaneous reports to detect previously unknown risks of vaccines, but uncertainty remains about the behavior of alternative epidemiologic designs to detect and declare a true risk early.Methods: Using three claims and one EHR database, we evaluate several variants of the case-control, comparative cohort, historical comparator, and self-controlled designs against historical vaccinations using real negative control outcomes (outcomes with no evidence to suggest that they could be caused by the vaccines) and simulated positive control outcomes.Results: Most methods show large type 1 error, often identifying false positive signals. The cohort method appears either positively or negatively biased, depending on the choice of comparator index date. Empirical calibration using effect-size estimates for negative control outcomes can bring type 1 error closer to nominal, often at the cost of increasing type 2 error. After calibration, the self-controlled case series (SCCS) design most rapidly detects small true effect sizes, while the historical comparator performs well for strong effects.Conclusion: When applying any method for vaccine safety surveillance we recommend considering the potential for systematic error, especially due to confounding, which for many designs appears to be substantial. Adjusting for age and sex alone is likely not sufficient to address differences between vaccinated and unvaccinated, and for the cohort method the choice of index date is important for the comparability of the groups. Analysis of negative control outcomes allows both quantification of the systematic error and, if desired, subsequent empirical calibration to restore type 1 error to its nominal value. In order to detect weaker signals, one may have to accept a higher type 1 error.

Objectives We propose a one-shot, privacy-preserving distributed algorithm to perform logistic regression (ODAL) across multiple clinical sites. Materials and Methods ODAL effectively utilizes the information from the local site (where the patient-level data are accessible) and incorporates the first-order (ODAL1) and second-order (ODAL2) gradients of the likelihood function from other sites to construct an estimator without requiring iterative communication across sites or transferring patient-level data. We evaluated ODAL via extensive simulation studies and an application to a dataset from the University of Pennsylvania Health System. The estimation accuracy was evaluated by comparing it with the estimator based on the combined individual participant data or pooled data (ie, gold standard). Results Our simulation studies revealed that the relative estimation bias of ODAL1 compared with the pooled estimates was Conclusions This study demonstrates that ODAL is privacy-preserving and communication-efficient with small bias and high statistical efficiency.

Background: Observational health data has the potential to be a rich resource to inform clinical practice and regulatory decision making. However, the lack of standard data quality processes makes it difficult to know if these data are research ready. The EHDEN COVID-19 Rapid Col-laboration Call presented the opportunity to assess how the newly developed open-source tool Data Quality Dashboard (DQD) informs the quality of data in a federated network. Methods: 15 Data Partners (DPs) from 10 different countries worked with the EHDEN taskforce to map their data to the OMOP CDM. Throughout the process at least two DQD results were collected and compared for each DP. Results: All DPs showed an improvement in their data quality between the first and last run of the DQD. The DQD excelled at helping DPs identify and fix conformance is-sues but showed less of an impact on completeness and plausibility checks. Conclusions: This is the first study to apply the DQD on multiple, disparate databases across a network. While study-specific checks should still be run, we recommend that all data holders converting their data to the OMOP CDM use the DQD as it ensures conformance to the model specifications and that a database meets a baseline level of completeness and plausibility for use in research.

Studies of the effects of medical interventions increasingly take place in distributed research settings using data from multiple clinical data sources including electronic health records and administrative claims. In such settings, privacy concerns typically prohibit sharing of individual patient data, and instead, cross-network analyses can only utilize summary statistics from the individual databases such as hazard ratios and standard errors. In the specific but very common context of the Cox proportional hazards model, we show that combining such per site summary statistics into a single network-wide estimate using standard meta-analysis methods leads to substantial bias when outcome counts are small. This bias derives primarily from the normal approximations of the per site likelihood that the methods utilized. Here we propose and evaluate methods that eschew normal approximations in favor of three more flexible approximations: a skew-normal, a one-dimensional grid, and a custom parametric function that mimics the behavior of the Cox likelihood function. In extensive simulation studies, we demonstrate how these approximations impact bias in the context of both fixed-effects and (Bayesian) random-effects models. We then apply these approaches to three real-world studies of the comparative safety of antidepressants, each using data from four observational health care databases.

Author(s): Khalid, Sara; Yang, Cynthia; Blacketer, Clair; Duarte-Salles, Talita; Fernandez-Bertolin, Sergio; Kim, Chungsoo; Park, Rae Woong; Park, Jimyung; Schuemie, Martijn J; Sena, Anthony G; Suchard, Marc A; You, Seng Chan; Rijnbeek, Peter R; Reps, Jenna M | Abstract: Background and objectiveAs a response to the ongoing COVID-19 pandemic, several prediction models in the existing literature were rapidly developed, with the aim of providing evidence-based guidance. However, none of these COVID-19 prediction models have been found to be reliable. Models are commonly assessed to have a risk of bias, often due to insufficient reporting, use of non-representative data, and lack of large-scale external validation. In this paper, we present the Observational Health Data Sciences and Informatics (OHDSI) analytics pipeline for patient-level prediction modeling as a standardized approach for rapid yet reliable development and validation of prediction models. We demonstrate how our analytics pipeline and open-source software tools can be used to answer important prediction questions while limiting potential causes of bias (e.g., by validating phenotypes, specifying the target population, performing large-scale external validation, and publicly providing all analytical source code).MethodsWe show step-by-step how to implement the analytics pipeline for the question: 'In patients hospitalized with COVID-19, what is the risk of death 0 to 30 days after hospitalization?'. We develop models using six different machine learning methods in a USA claims database containing over 20,000 COVID-19 hospitalizations and externally validate the models using data containing over 45,000 COVID-19 hospitalizations from South Korea, Spain, and the USA.ResultsOur open-source software tools enabled us to efficiently go end-to-end from problem design to reliable Model Development and evaluation. When predicting death in patients hospitalized with COVID-19, AdaBoost, random forest, gradient boosting machine, and decision tree yielded similar or lower internal and external validation discrimination performance compared to L1-regularized logistic regression, whereas the MLP neural network consistently resulted in lower discrimination. L1-regularized logistic regression models were well calibrated.ConclusionOur results show that following the OHDSI analytics pipeline for patient-level prediction modelling can enable the rapid development towards reliable prediction models. The OHDSI software tools and pipeline are open source and available to researchers from all around the world.

IntroductionObservational data enables large-scale vaccine safety surveillance but requires careful evaluation of potential sources of bias. One potential source of bias is an index date selection procedure for the unvaccinated cohort or unvaccinated comparison time. Here, we evaluate different index date selection procedures for two vaccines: COVID-19 and influenza.MethodsFor each vaccine, we extracted patient baseline characteristics on the index date and up to 450 days prior and then compared them to the characteristics of the unvaccinated patients indexed on an arbitrary date or indexed on a date of a visit. Additionally, we compared vaccinated patients indexed on the date of vaccination and the same patients indexed on a prior date or visit.ResultsCOVID-19 vaccination and influenza vaccination differ drastically from each other in terms of populations vaccinated and their status on the day of vaccination. When compared to indexing on a visit in unvaccinated population, influenza vaccination had markedly higher covariate proportions and COVID-19 vaccination had lower proportions of most covariates on the index date. In contrast, COVID-19 vaccination had similar covariate proportions when compared to an arbitrary date. These effects attenuated but were still present with a longer lookback period. The effect of day 0 was present even when patients served as their own controls.ConclusionPatient baseline characteristics are sensitive to the choice of the index date. In vaccine safety studies, unexposed index event should represent vaccination settings. Study designs previously used to assess influenza vaccination must be reassessed for COVID-19 to account for a potentially healthier population and lack of medical activity on the day of vaccination.

Refereed/Peer-reviewed Using real-world data and past vaccination data, we conducted a large-scale experiment to quantify bias, precision and timeliness of different study designs to estimate historical background (expected) compared to post-vaccination (observed) rates of safety events for several vaccines. We used negative (not causally related) and positive control outcomes. The latter were synthetically generated true safety signals with incident rate ratios ranging from 1.5 to 4. Observed vs. expected analysis using within-database historical background rates is a sensitive but unspecific method for the identification of potential vaccine safety signals. Despite good discrimination, most analyses showed a tendency to overestimate risks, with 20%-100% type 1 error, but low (0% to 20%) type 2 error in the large databases included in our study. Efforts to improve the comparability of background and post-vaccine rates, including age-sex adjustment and anchoring background rates around a visit, reduced type 1 error and improved precision but residual systematic error persisted. Additionally, empirical calibration dramatically reduced type 1 to nominal but came at the cost of increasing type 2 error.

Background Observational data enables large-scale vaccine safety surveillance but requires careful evaluation of the potential sources of bias. One potential source of bias is the index date selection procedure for the unvaccinated cohort or unvaccinated comparison time (“anchoring”). Objective Here, we evaluated the different index date selection procedures for 2 vaccinations: COVID-19 and influenza. Methods For each vaccine, we extracted patient baseline characteristics on the index date and up to 450 days prior and then compared them to the characteristics of the unvaccinated patients indexed on (1) an arbitrary date or (2) a date of a visit. Additionally, we compared vaccinated patients indexed on the date of vaccination and the same patients indexed on a prior date or visit. Results COVID-19 vaccination and influenza vaccination differ drastically from each other in terms of the populations vaccinated and their status on the day of vaccination. When compared to indexing on a visit in the unvaccinated population, influenza vaccination had markedly higher covariate proportions, and COVID-19 vaccination had lower proportions of most covariates on the index date. In contrast, COVID-19 vaccination had similar covariate proportions when compared to an arbitrary date. These effects attenuated, but were still present, with a longer lookback period. The effect of day 0 was present even when the patients served as their own controls. Conclusions Patient baseline characteristics are sensitive to the choice of the index date. In vaccine safety studies, unexposed index event should represent vaccination settings. Study designs previously used to assess influenza vaccination must be reassessed for COVID-19 to account for a potentially healthier population and lack of medical activity on the day of vaccination.

BACKGROUND Observational data enables large-scale vaccine safety surveillance but requires careful evaluation of the potential sources of bias. One potential source of bias is the index date selection procedure for the unvaccinated cohort or unvaccinated comparison time (“anchoring”). OBJECTIVE Here, we evaluated the different index date selection procedures for 2 vaccinations: COVID-19 and influenza. METHODS For each vaccine, we extracted patient baseline characteristics on the index date and up to 450 days prior and then compared them to the characteristics of the unvaccinated patients indexed on (1) an arbitrary date or (2) a date of a visit. Additionally, we compared vaccinated patients indexed on the date of vaccination and the same patients indexed on a prior date or visit. RESULTS COVID-19 vaccination and influenza vaccination differ drastically from each other in terms of the populations vaccinated and their status on the day of vaccination. When compared to indexing on a visit in the unvaccinated population, influenza vaccination had markedly higher covariate proportions, and COVID-19 vaccination had lower proportions of most covariates on the index date. In contrast, COVID-19 vaccination had similar covariate proportions when compared to an arbitrary date. These effects attenuated, but were still present, with a longer lookback period. The effect of day 0 was present even when the patients served as their own controls. CONCLUSIONS Patient baseline characteristics are sensitive to the choice of the index date. In vaccine safety studies, unexposed index event should represent vaccination settings. Study designs previously used to assess influenza vaccination must be reassessed for COVID-19 to account for a potentially healthier population and lack of medical activity on the day of vaccination.

Objective: The current observational research literature shows extensive publication bias and contradiction. The Observational Health Data Sciences and Informatics (OHDSI) initiative seeks to improve research reproducibility through open science. Methods: OHDSI has created an international federated data source of electronic health records and administrative claims that covers nearly 10% of the world’s population. Using a common data model with a practical schema and extensive vocabulary mappings, data from around the world follow the identical format. OHDSI’s research methods emphasize reproducibility, with a large-scale approach to addressing confounding using propensity score adjustment with extensive diagnostics; negative and positive control hypotheses to test for residual systematic error; a variety of data sources to assess consistency and generalizability; a completely open approach including protocol, software, models, parameters, and raw results so that studies can be externally verified; and the study of many hypotheses in parallel so that the operating characteristics of the methods can be assessed. Results: OHDSI has already produced findings in areas like hypertension treatment that are being incorporated into practice, and it has produced rigorous studies of COVID-19 that have aided government agencies in their treatment decisions, that have characterized the disease extensively, that have estimated the comparative effects of treatments, and that the predict likelihood of advancing to serious complications. Conclusions: OHDSI practices open science and incorporates a series of methods to address reproducibility. It has produced important results in several areas, including hypertension therapy and COVID-19 research.

Purpose To compare the incidence of diabetic ketoacidosis (DKA) among patients with type 2 diabetes mellitus (T2DM) who were new users of sodium glucose co‐transporter 2 inhibitors (SGLT2i) versus other classes of antihyperglycemic agents (AHAs). Methods Patients were identified from four large US claims databases using broad (all T2DM patients) and narrow (intended to exclude patients with type 1 diabetes or secondary diabetes misclassified as T2DM) definitions of T2DM. New users of SGLT2i and seven groups of comparator AHAs were matched (1:1) on exposure propensity scores to adjust for imbalances in baseline covariates. Cox proportional hazards regression models, conditioned on propensity score‐matched pairs, were used to estimate hazard ratios (HRs) of DKA for new users of SGLT2i versus other AHAs. When I2

IntroductionTherapeutic options for type 2 diabetes mellitus (T2DM) have expanded over the last decade with the emergence of cardioprotective novel agents, but without such data for older drugs, leaving a critical gap in our understanding of the relative effects of T2DM agents on cardiovascular risk.Methods and analysisThe large-scale evidence generations across a network of databases for T2DM (LEGEND-T2DM) initiative is a series of systematic, large-scale, multinational, real-world comparative cardiovascular effectiveness and safety studies of all four major second-line anti-hyperglycaemic agents, including sodium–glucose co-transporter-2 inhibitor, glucagon-like peptide-1 receptor agonist, dipeptidyl peptidase-4 inhibitor and sulfonylureas. LEGEND-T2DM will leverage the Observational Health Data Sciences and Informatics (OHDSI) community that provides access to a global network of administrative claims and electronic health record data sources, representing 190 million patients in the USA and about 50 million internationally. LEGEND-T2DM will identify all adult, patients with T2DM who newly initiate a traditionally second-line T2DM agent. Using an active comparator, new-user cohort design, LEGEND-T2DM will execute all pairwise class-versus-class and drug-versus-drug comparisons in each data source, producing extensive study diagnostics that assess reliability and generalisability through cohort balance and equipoise to examine the relative risk of cardiovascular and safety outcomes. The primary cardiovascular outcomes include a composite of major adverse cardiovascular events and a series of safety outcomes. The study will pursue data-driven, large-scale propensity adjustment for measured confounding, a large set of negative control outcome experiments to address unmeasured and systematic bias.Ethics and disseminationThe study ensures data safety through a federated analytic approach and follows research best practices, including prespecification and full disclosure of results. LEGEND-T2DM is dedicated to open science and transparency and will publicly share all analytic code from reproducible cohort definitions through turn-key software, enabling other research groups to leverage our methods, data and results to verify and extend our findings.

Background Cardinality matching (CM), a novel matching technique, finds the largest matched sample meeting prespecified balance criteria thereby overcoming limitations of propensity score matching (PSM) associated with limited covariate overlap, which are especially pronounced in studies with small sample sizes. The current study proposes a framework for large-scale CM (LS-CM); and compares large-scale PSM (LS-PSM) and LS-CM in terms of post-match sample size, covariate balance and residual confounding at progressively smaller sample sizes. Methods Evaluation of LS-PSM and LS-CM within a comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and thiazide or thiazide-like diuretic monotherapy identified from a U.S. insurance claims database. Candidate covariates included patient demographics, and all observed prior conditions, drug exposures and procedures. Propensity scores were calculated using LASSO regression, and candidate covariates with non-zero beta coefficients in the propensity model were defined as matching covariates for use in LS-CM. One-to-one matching was performed using progressively tighter parameter settings. Covariate balance was assessed using standardized mean differences. Hazard ratios for negative control outcomes perceived as unassociated with treatment (i.e., true hazard ratio of 1) were estimated using unconditional Cox models. Residual confounding was assessed using the expected systematic error of the empirical null distribution of negative control effect estimates compared to the ground truth. To simulate diverse research conditions, analyses were repeated within 10 %, 1 and 0.5 % subsample groups with increasingly limited covariate overlap. Results A total of 172,117 patients (ACEI: 129,078; thiazide: 43,039) met the study criteria. As compared to LS-PSM, LS-CM was associated with increased sample retention. Although LS-PSM achieved balance across all matching covariates within the full study population, substantial matching covariate imbalance was observed within the 1 and 0.5 % subsample groups. Meanwhile, LS-CM achieved matching covariate balance across all analyses. LS-PSM was associated with better candidate covariate balance within the full study population. Otherwise, both matching techniques achieved comparable candidate covariate balance and expected systematic error. Conclusions LS-CM found the largest matched sample meeting prespecified balance criteria while achieving comparable candidate covariate balance and residual confounding. We recommend LS-CM as an alternative to LS-PSM in studies with small sample sizes or limited covariate overlap.

Linear mixed models are commonly used in healthcare-based association analyses for analyzing multi-site data with heterogeneous site-specific random effects. Due to regulations for protecting patients’ privacy, sensitive individual patient data (IPD) typically cannot be shared across sites. We propose an algorithm for fitting distributed linear mixed models (DLMMs) without sharing IPD across sites. This algorithm achieves results identical to those achieved using pooled IPD from multiple sites (i.e., the same effect size and standard error estimates), hence demonstrating the lossless property. The algorithm requires each site to contribute minimal aggregated data in only one round of communication. We demonstrate the lossless property of the proposed DLMM algorithm by investigating the associations between demographic and clinical characteristics and length of hospital stay in COVID-19 patients using administrative claims from the UnitedHealth Group Clinical Discovery Database. We extend this association study by incorporating 120,609 COVID-19 patients from 11 collaborative data sources worldwide.

Alpha-1 blockers, often used to treat benign prostate hyperplasia (BPH), have been hypothesized to prevent COVID-19 complications by minimising cytokine storms release. We conducted a prevalent-user active-comparator cohort study to assess association between alpha-1 blocker use and risks of three COVID-19 outcomes: diagnosis, hospitalization, and hospitalization requiring intensive services. Our study included 2.6 and 0.46 million users of alpha-1 blockers and of alternative BPH therapy during the period between November 2019 and January 2020, found in electronic health records from Spain (SIDIAP) and the United States (Department of Veterans Affairs, Columbia University Irving Medical Center, IQVIA OpenClaims, Optum DOD, Optum EHR). We estimated hazard ratios using state-of-the-art techniques to minimize potential confounding, including large-scale propensity score matching/stratification and negative control calibration. We found no differential risk for any of COVID-19 outcome, pointing to the need for further research on potential COVID-19 therapies.

BackgroundThe design used to create labelled data for training prediction models from observational healthcare databases (e.g., case-control and cohort) may impact the clinical usefulness. We aim to investigate hypothetical design issues and determine how the design impacts prediction model performance.AimTo empirically investigate differences between models developed using a case-control design and a cohort design.MethodsUsing a US claims database, we replicated two published prediction models (dementia and type 2 diabetes) which were developed using a case-control design, and trained models for the same prediction questions using cohort designs. We validated each model on data mimicking the point in time the models would be applied in clinical practice. We calculated the models’ discrimination and calibration-in-the-large performances.ResultsThe dementia models obtained area under the receiver operating characteristics of 0.560 and 0.897 for the case-control and cohort designs respectively. The type 2 diabetes models obtained area under the receiver operating characteristics of 0.733 and 0.727 for the case-control and cohort designs respectively. The dementia and diabetes case-control models were both poorly calibrated, whereas the dementia cohort model achieved good calibration. We show that careful construction of a case-control design can lead to comparable discriminative performance as a cohort design, but case-control designs over-represent the outcome class leading to miscalibration.ConclusionsAny case-control design can be converted to a cohort design. We recommend that researchers with observational data use the less subjective and generally better calibrated cohort design when extracting labelled data. However, if a carefully constructed case-control design is used, then the model must be prospectively validated using a cohort design for fair evaluation and be recalibrated.

ACE (angiotensin-converting enzyme) inhibitors and angiotensin receptor blockers (ARBs) are equally guideline-recommended first-line treatments for hypertension, yet few head-to-head studies exist. We compared the real-world effectiveness and safety of ACE inhibitors versus ARBs in the first-line treatment of hypertension. We implemented a retrospective, new-user comparative cohort design to estimate hazard ratios using techniques to minimize residual confounding and bias, specifically large-scale propensity score adjustment, empirical calibration, and full transparency. We included all patients with hypertension initiating monotherapy with an ACE inhibitor or ARB between 1996 and 2018 across 8 databases from the United States, Germany, and South Korea. The primary outcomes were acute myocardial infarction, heart failure, stroke, and composite cardiovascular events. We also studied 51 secondary and safety outcomes including angioedema, cough, syncope, and electrolyte abnormalities. Across 8 databases, we identified 2 297 881 patients initiating treatment with ACE inhibitors and 673 938 patients with ARBs. We found no statistically significant difference in the primary outcomes of acute myocardial infarction (hazard ratio, 1.11 for ACE versus ARB [95% CI, 0.95-1.32]), heart failure (hazard ratio, 1.03 [0.87-1.24]), stroke (hazard ratio, 1.07 [0.91-1.27]), or composite cardiovascular events (hazard ratio, 1.06 [0.90-1.25]). Across secondary and safety outcomes, patients on ARBs had significantly lower risk of angioedema, cough, pancreatitis, and GI bleeding. In our large-scale, observational network study, ARBs do not differ statistically significantly in effectiveness at the class level compared with ACE inhibitors as first-line treatment for hypertension but present a better safety profile. These findings support preferentially prescribing ARBs over ACE inhibitors when initiating treatment for hypertension. Refereed/Peer-reviewed

Linear mixed models (LMMs) are commonly used in many areas including epidemiology for analyzing multi-site data with heterogeneous site-specific random effects. However, due to the regulation of protecting patients’ privacy, sensitive individual patient data (IPD) are usually not allowed to be shared across sites. In this paper we propose a novel algorithm for distributed linear mixed models (DLMMs). Our proposed DLMM algorithm can achieve exactly the same results as if we had pooled IPD from all sites, hence the lossless property. The DLMM algorithm requires each site to contribute some aggregated data (AD) in only one iteration. We apply the proposed DLMM algorithm to analyze the association of length of stay of COVID-19 hospitalization with demographic and clinical characteristics using the administrative claims database from the UnitedHealth Group Clinical Research Database.

Background: Cardinality matching (CM), a novel matching technique, finds the largest matched sample meeting prespecified balance criteria thereby overcoming limitations of propensity score matching (PSM) associated with limited covariate overlap, which are especially pronounced in studies with small sample sizes. The current study compares CM and PSM in terms of post-match sample size, covariate balance and residual confounding at progressively smaller sample sizes.Methods: To evaluate CM and PSM within a comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and thiazide or thiazide-like diuretic monotherapy identified from a U.S. insurance claims database. Candidate covariates included patient demographics, and all observed prior conditions, drug exposures and procedures. Propensity scores were calculated using LASSO regression, and candidate covariates with non-zero beta coefficients in the propensity model were defined as matching covariates for use in CM. One-to-one matching was performed using progressively tighter parameter settings. Covariate balance was assessed using standardized mean differences. Hazard ratios were estimated using unconditional Cox models for negative control outcomes perceived as unassociated with treatment (e.g., true hazard ratio of 1). Residual confounding was assessed using the expected systematic error of the empirical null distribution of negative control effect estimates compared to the ground truth. Analyses were repeated within 10%, 1% and 0.5% subsample groups.Results: A total of 172,117 patients (ACEI: 129,078; thiazide: 43,039) met the study criteria. Compared to PSM, CM was associated with increased sample retention except for analyses failing to converge to a matched sample. Although PSM achieved balance across all matching covariates within the full study population, substantial matching covariate imbalance was observed within the 1% and 0.5% subsample groups. Meanwhile, CM achieved matching covariate balance across all analyses. PSM was associated with better candidate covariate balance within the full study population. Otherwise, both matching techniques achieved comparable candidate covariate balance and expected systematic error.Conclusion: CM found the largest matched sample meeting prespecified balance criteria while achieving comparable candidate covariate balance and residual confounding. At smaller sample sizes, CM achieved superior matching covariate balance. We recommend CM as an alternative to PSM in studies with small sample sizes.

Background To better understand the profile of individuals with severe coronavirus disease 2019 (COVID-19), we characterised individuals hospitalised with COVID-19 and compared them to individuals previously hospitalised with influenza. Methods We report the characteristics (demographics, prior conditions and medication use) of patients hospitalised with COVID-19 between December 2019 and April 2020 in the US (Columbia University Irving Medical Center [CUIMC], STAnford Medicine Research data Repository [STARR-OMOP], and the Department of Veterans Affairs [VA OMOP]) and Health Insurance Review & Assessment [HIRA] of South Korea. Patients hospitalised with COVID-19 were compared with patients previously hospitalised with influenza in 2014–19. Results 6,806 (US: 1,634, South Korea: 5,172) individuals hospitalised with COVID-19 were included. Patients in the US were majority male (VA OMOP: 94%, STARR-OMOP: 57%, CUIMC: 52%), but were majority female in HIRA (56%). Age profiles varied across data sources. Prevalence of asthma ranged from 7% to 14%, diabetes from 18% to 43%, and hypertensive disorder from 22% to 70% across data sources, while between 9% and 39% were taking drugs acting on the renin-angiotensin system in the 30 days prior to their hospitalisation. Compared to 52,422 individuals hospitalised with influenza, patients admitted with COVID-19 were more likely male, younger, and, in the US, had fewer comorbidities and lower medication use. Conclusions Rates of comorbidities and medication use are high among individuals hospitalised with COVID-19. However, COVID-19 patients are more likely to be male and appear to be younger and, in the US, generally healthier than those typically admitted with influenza.

Background: Researchers developing prediction models are faced with numerous design choices that may impact model performance. One of the main decisions is how to include patients who are lost to follow-up. In this paper we perform a large-scale empirical evaluation investigating the impact of this decision. In addition, we aim to provide guidelines for how to deal with loss to follow-up. Methods: We generate a synthetic dataset with complete follow-up and simulate loss to follow-up based either on random selection or on selection based on comorbidity. We investigate four simple strategies for developing models using data containing some patients with loss to follow-up. Three strategies employ a binary classifier with data that: i) include all patients (including those lost to follow-up), ii) exclude all patients lost to follow-up or iii) only exclude patients lost to follow-up who do not have the outcome before being lost to follow-up. The fourth strategy uses a survival model with data that include all patients. In addition to our synthetic data study, we empirically evaluate the discrimination and calibration performance of these strategies across 21 prediction problems using real-world data. Results: The synthetic data study results show that excluding patients lost to follow-up can introduce bias when loss to follow-up is common and does not occur at random. However, when loss to follow-up was completely at random, the choice of addressing it had negligible impact on the model performance. Our empirical results showed that the four design choices investigated to deal with loss to follow-up resulted in comparable performance when the time-at-risk was 1-year, but demonstrated differential bias when we looking into 3-year time-at-risk. Removing patients who are lost to follow-up before the outcome but keeping patients who are loss to follow-up after the outcome can bias a model and should be avoided. Conclusion: Based on this study we therefore recommend i) developing models using data that includes patients that are lost to follow-up and ii) evaluate the discrimination and calibration of models twice: on a test set including patients lost to follow-up and a test set excluding patients lost to follow-up.

Alendronate and raloxifene are among the most popular anti-osteoporosis medications. However, there is a lack of head-to-head comparative effectiveness studies comparing the two treatments. We conducted a retrospective large-scale multicenter study encompassing over 300 million patients across nine databases encoded in the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM). The primary outcome was the incidence of osteoporotic hip fracture, while secondary outcomes were vertebral fracture, atypical femoral fracture (AFF), osteonecrosis of the jaw (ONJ), and esophageal cancer. We used propensity score trimming and stratification based on an expansive propensity score model with all pre-treatment patient characteritistcs. We accounted for unmeasured confounding using negative control outcomes to estimate and adjust for residual systematic bias in each data source. We identified 283,586 alendronate patients and 40,463 raloxifene patients. There were 7.48 hip fracture, 8.18 vertebral fracture, 1.14 AFF, 0.21 esophageal cancer and 0.09 ONJ events per 1,000 person-years in the alendronate cohort and 6.62, 7.36, 0.69, 0.22 and 0.06 events per 1,000 person-years, respectively, in the raloxifene cohort. Alendronate and raloxifene have a similar hip fracture risk (hazard ratio [HR] 1.03, 95% confidence interval [CI] 0.94–1.13), but alendronate users are more likely to have vertebral fractures (HR 1.07, 95% CI 1.01–1.14). Alendronate has higher risk for AFF (HR 1.51, 95% CI 1.23–1.84) but similar risk for esophageal cancer (HR 0.95, 95% CI 0.53–1.70), and ONJ (HR 1.62, 95% CI 0.78–3.34). We demonstrated substantial control of measured confounding by propensity score adjustment, and minimal residual systematic bias through negative control experiments, lending credibility to our effect estimates. Raloxifene is as effective as alendronate and may remain an option in the prevention of osteoporotic fracture.

Comorbid conditions appear to be common among individuals hospitalised with coronavirus disease 2019 (COVID-19) but estimates of prevalence vary and little is known about the prior medication use of patients. Here, we describe the characteristics of adults hospitalised with COVID-19 and compare them with influenza patients. We include 34,128 (US: 8362, South Korea: 7341, Spain: 18,425) COVID-19 patients, summarising between 4811 and 11,643 unique aggregate characteristics. COVID-19 patients have been majority male in the US and Spain, but predominantly female in South Korea. Age profiles vary across data sources. Compared to 84,585 individuals hospitalised with influenza in 2014-19, COVID-19 patients have more typically been male, younger, and with fewer comorbidities and lower medication use. While protecting groups vulnerable to influenza is likely a useful starting point in the response to COVID-19, strategies will likely need to be broadened to reflect the particular characteristics of individuals being hospitalised with COVID-19., Detailed knowledge of the characteristics of COVID-19 patients helps with public health planning. Here, the authors use routinely-collected data from seven databases in three countries to describe the characteristics of >30,000 patients admitted with COVID-19 and compare them with those admitted for influenza in previous years.

BackgroundHydroxychloroquine has recently received Emergency Use Authorization by the FDA and is currently prescribed in combination with azithromycin for COVID-19 pneumonia. We studied the safety of hydroxychloroquine, alone and in combination with azithromycin.MethodsNew user cohort studies were conducted including 16 severe adverse events (SAEs). Rheumatoid arthritis patients aged 18+ and initiating hydroxychloroquine were compared to those initiating sulfasalazine and followed up over 30 days. Self-controlled case series (SCCS) were conducted to further establish safety in wider populations. Separately, SAEs associated with hydroxychloroquine- azithromycin (compared to hydroxychloroquine-amoxicillin) were studied. Data comprised 14 sources of claims data or electronic medical records from Germany, Japan, Netherlands, Spain, UK, and USA. Propensity score stratification and calibration using negative control outcomes were used to address confounding. Cox models were fitted to estimate calibrated hazard ratios (CalHRs) according to drug use. Estimates were pooled where I2ResultsOverall, 956,374 and 310,350 users of hydroxychloroquine and sulfasalazine, and 323,122 and 351,956 users of hydroxychloroquine-azithromycin and hydroxychloroquine-amoxicillin were included. No excess risk of SAEs was identified when 30-day hydroxychloroquine and sulfasalazine use were compared. SCCS confirmed these findings. However, when azithromycin was added to hydroxychloroquine, we observed an increased risk of 30-day cardiovascular mortality (CalHR2.19 [1.22- 3.94]), chest pain/angina (CalHR 1.15 [95% CI 1.05-1.26]), and heart failure (CalHR 1.22 [95% CI 1.02- 1.45])ConclusionsShort-term hydroxychloroquine treatment is safe, but addition of azithromycin may induce heart failure and cardiovascular mortality, potentially due to synergistic effects on QT length. We call for caution if such combination is to be used in the management of Covid-19.Trial registration numberRegistered with EU PAS; Reference number EUPAS34497 (http://www.encepp.eu/encepp/viewResource.htm?id=34498). The full study protocol and analysis source code can be found at https://github.com/ohdsi-studies/Covid19EstimationHydroxychloroquine.Funding sourcesThis research received partial support from the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre (BRC) and Senior Research Fellowship (DPA), US National Institutes of Health, Janssen Research & Development, IQVIA, and by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea [grant number: HI16C0992]. Personal funding included Versus Arthritis [21605] (JL), MRC-DTP [MR/K501256/1] (JL), MRC and FAME (APU). The European Health Data & Evidence Network has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement No 806968. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. No funders had a direct role in this study. The views and opinions expressed are those of the authors and do not necessarily reflect those of the Clinician Scientist Award programme, NIHR, NHS or the Department of Health, England.

Objective We developed and evaluated a privacy-preserving One-shot Distributed Algorithm to fit a multicenter Cox proportional hazards model (ODAC) without sharing patient-level information across sites. Materials and Methods Using patient-level data from a single site combined with only aggregated information from other sites, we constructed a surrogate likelihood function, approximating the Cox partial likelihood function obtained using patient-level data from all sites. By maximizing the surrogate likelihood function, each site obtained a local estimate of the model parameter, and the ODAC estimator was constructed as a weighted average of all the local estimates. We evaluated the performance of ODAC with (1) a simulation study and (2) a real-world use case study using 4 datasets from the Observational Health Data Sciences and Informatics network. Results On the one hand, our simulation study showed that ODAC provided estimates nearly the same as the estimator obtained by analyzing, in a single dataset, the combined patient-level data from all sites (ie, the pooled estimator). The relative bias was Conclusions ODAC is a privacy-preserving and noniterative method for implementing time-to-event analyses across multiple sites. It provides estimates on par with the pooled estimator and substantially outperforms the meta-analysis estimator when the event is uncommon, making it extremely suitable for studying rare events and diseases in a distributed manner.

Importance Chlorthalidone is currently recommended as the preferred thiazide diuretic to treat hypertension, but no trials have directly compared risks and benefits. Objective To compare the effectiveness and safety of chlorthalidone and hydrochlorothiazide as first-line therapies for hypertension in real-world practice. Design, Setting, and Participants This is a Large-Scale Evidence Generation and Evaluation in a Network of Databases (LEGEND) observational comparative cohort study with large-scale propensity score stratification and negative-control and synthetic positive-control calibration on databases spanning January 2001 through December 2018. Outpatient and inpatient care episodes of first-time users of antihypertensive monotherapy in the United States based on 2 administrative claims databases and 1 collection of electronic health records were analyzed. Analysis began June 2018. Exposures Chlorthalidone and hydrochlorothiazide. Main Outcomes and Measures The primary outcomes were acute myocardial infarction, hospitalization for heart failure, ischemic or hemorrhagic stroke, and a composite cardiovascular disease outcome including the first 3 outcomes and sudden cardiac death. Fifty-one safety outcomes were measured. Results Of 730 225 individuals (mean [SD] age, 51.5 [13.3] years; 450 100 women [61.6%]), 36 918 were dispensed or prescribed chlorthalidone and had 149 composite outcome events, and 693 337 were dispensed or prescribed hydrochlorothiazide and had 3089 composite outcome events. No significant difference was found in the associated risk of myocardial infarction, hospitalized heart failure, or stroke, with a calibrated hazard ratio for the composite cardiovascular outcome of 1.00 for chlorthalidone compared with hydrochlorothiazide (95% CI, 0.85-1.17). Chlorthalidone was associated with a significantly higher risk of hypokalemia (hazard ratio [HR], 2.72; 95% CI, 2.38-3.12), hyponatremia (HR, 1.31; 95% CI, 1.16-1.47), acute renal failure (HR, 1.37; 95% CI, 1.15-1.63), chronic kidney disease (HR, 1.24; 95% CI, 1.09-1.42), and type 2 diabetes mellitus (HR, 1.21; 95% CI, 1.12-1.30). Chlorthalidone was associated with a significantly lower risk of diagnosed abnormal weight gain (HR, 0.73; 95% CI, 0.61-0.86). Conclusions and Relevance This study found that chlorthalidone use was not associated with significant cardiovascular benefits when compared with hydrochlorothiazide, while its use was associated with greater risk of renal and electrolyte abnormalities. These findings do not support current recommendations to prefer chlorthalidone vs hydrochlorothiazide for hypertension treatment in first-time users was found. We used advanced methods, sensitivity analyses, and diagnostics, but given the possibility of residual confounding and the limited length of observation periods, further study is warranted.

Healthcare professionals increasingly rely on observational healthcare data, such as administrative claims and electronic health records, to estimate the causal effects of interventions. However, limited prior studies raise concerns about the real-world performance of the statistical and epidemiological methods that are used. We present the “OHDSI Methods Benchmark” that aims to evaluate the performance of effect estimation methods on real data. The benchmark comprises a gold standard, a set of metrics, and a set of open source software tools. The gold standard is a collection of real negative controls (drug-outcome pairs where no causal effect appears to exist) and synthetic positive controls (drug-outcome pairs that augment negative controls with simulated causal effects). We apply the benchmark using four large healthcare databases to evaluate methods commonly used in practice: the new-user cohort, self-controlled cohort, case-control, case-crossover, and self-controlled case series designs. The results confirm the concerns about these methods, showing that for most methods the operating characteristics deviate considerably from nominal levels. For example, in most contexts, only half of the 95% confidence intervals we calculated contain the corresponding true effect size. We previously developed an “empirical calibration” procedure to restore these characteristics and we also evaluate this procedure. While no one method dominates, self-controlled methods such as the empirically calibrated self-controlled case series perform well across a wide range of scenarios.

This study sought to: 1. determine the frequency of rehospitalization with diagnosis of suicidal ideation or suicide attempt (SI/SA) within a year and how often patients had multiple rehospitalizations; 2. identify the time period for which the risk of rehospitalization is highest; and 3. determine the characteristics of patients with multiple rehospitalizations. We conducted a retrospective cohort study of adults with depression using 4 US health claims databases. We defined hospitalization as an inpatient or emergency room visit with codes indicating a suicide attempt or suicidal thoughts using a validated algorithm. Rates of rehospitalization with SI or SA were analyzed together and separately, including multiple re-hospitalizations with SI/SA. Across all databases 121,065 patients were hospitalized with a diagnosis of SI/SA. Rates of rehospitalization within a year ranged from 7.96% to 11.24%. The risk of rehospitalization with SI/SA is highest during the first month. Nearly 50% of rehospitalizations occurred within 3 months after initial hospitalization. Patients with rehospitalization(s) had more anxiety disorders, sleep disorders and substance use disorders than patients without. Among patients with depression hospitalized for SI/SA, rehospitalization for SI/SA within a year is not uncommon. Risk of rehospitalization with a diagnosis of SI/SA is highest during the first month.

ObjectiveTo measure the paediatric user and prescription prevalence in inpatient and ambulatory settings in South Korea, Hong Kong, Taiwan, Japan and Australia by age and gender. A further objective was to list the most commonly used drugs per drug class, per country.Design and settingHospital inpatient and insurance paediatric healthcare data from the following databases were used to conduct this descriptive drug utilisation study: (i) the South Korean Ajou University School of Medicine database; (ii) the Hong Kong Clinical Data Analysis and Reporting System; (iii) the Japan Medical Data Center; (iv) Taiwan’s National Health Insurance Research Database and (v) the Australian Pharmaceutical Benefits Scheme. Country-specific data were transformed into the Observational Medical Outcomes Partnership Common Data Model.PatientsChildren (≤18 years) with at least 1 day of observation in any of the respective databases from January 2009 until December 2013 were included.Main outcome measuresFor each drug class, we assessed the per-protocol overall user and prescription prevalence rates (per 1000 persons) per country and setting.ResultsOur study population comprised 1 574 524 children (52.9% male). The highest proportion of dispensings was recorded in the youngest age category (ConclusionsCountry-specific paediatric drug utilisation patterns were described, ranked and compared between four East Asian countries and Australia. The widespread use of mucolytics in East Asia warrants further investigation.

Background Hydroxychloroquine, a drug commonly used in the treatment of rheumatoid arthritis, has received much negative publicity for adverse events associated with its authorisation for emergency use to treat patients with COVID-19 pneumonia. We studied the safety of hydroxychloroquine, alone and in combination with azithromycin, to determine the risk associated with its use in routine care in patients with rheumatoid arthritis. Methods In this multinational, retrospective study, new user cohort studies in patients with rheumatoid arthritis aged 18 years or older and initiating hydroxychloroquine were compared with those initiating sulfasalazine and followed up over 30 days, with 16 severe adverse events studied. Self-controlled case series were done to further establish safety in wider populations, and included all users of hydroxychloroquine regardless of rheumatoid arthritis status or indication. Separately, severe adverse events associated with hydroxychloroquine plus azithromycin (compared with hydroxychloroquine plus amoxicillin) were studied. Data comprised 14 sources of claims data or electronic medical records from Germany, Japan, the Netherlands, Spain, the UK, and the USA. Propensity score stratification and calibration using negative control outcomes were used to address confounding. Cox models were fitted to estimate calibrated hazard ratios (HRs) according to drug use. Estimates were pooled where the I2 value was less than 0·4. Findings The study included 956 374 users of hydroxychloroquine, 310 350 users of sulfasalazine, 323 122 users of hydroxychloroquine plus azithromycin, and 351 956 users of hydroxychloroquine plus amoxicillin. No excess risk of severe adverse events was identified when 30-day hydroxychloroquine and sulfasalazine use were compared. Self-controlled case series confirmed these findings. However, long-term use of hydroxychloroquine appeared to be associated with increased cardiovascular mortality (calibrated HR 1·65 [95% CI 1·12–2·44]). Addition of azithromycin appeared to be associated with an increased risk of 30-day cardiovascular mortality (calibrated HR 2·19 [95% CI 1·22–3·95]), chest pain or angina (1·15 [1·05–1·26]), and heart failure (1·22 [1·02–1·45]). Interpretation Hydroxychloroquine treatment appears to have no increased risk in the short term among patients with rheumatoid arthritis, but in the long term it appears to be associated with excess cardiovascular mortality. The addition of azithromycin increases the risk of heart failure and cardiovascular mortality even in the short term. We call for careful consideration of the benefit–risk trade-off when counselling those on hydroxychloroquine treatment. Funding National Institute for Health Research (NIHR) Oxford Biomedical Research Centre, NIHR Senior Research Fellowship programme, US National Institutes of Health, US Department of Veterans Affairs, Janssen Research and Development, IQVIA, Korea Health Industry Development Institute through the Ministry of Health and Welfare Republic of Korea, Versus Arthritis, UK Medical Research Council Doctoral Training Partnership, Foundation Alfonso Martin Escudero, Innovation Fund Denmark, Novo Nordisk Foundation, Singapore Ministry of Health's National Medical Research Council Open Fund Large Collaborative Grant, VINCI, Innovative Medicines Initiative 2 Joint Undertaking, EU's Horizon 2020 research and innovation programme, and European Federation of Pharmaceutical Industries and Associations.

Objectives To demonstrate the application of the Large-scale Evidence Generation and Evaluation across a Network of Databases (LEGEND) principles described in our companion article to hypertension treatments and assess internal and external validity of the generated evidence. Materials and Methods LEGEND defines a process for high-quality observational research based on 10 guiding principles. We demonstrate how this process, here implemented through large-scale propensity score modeling, negative and positive control questions, empirical calibration, and full transparency, can be applied to compare antihypertensive drug therapies. We assess internal validity through covariate balance, confidence-interval coverage, between-database heterogeneity, and transitivity of results. We assess external validity through comparison to direct meta-analyses of randomized controlled trials (RCTs). Results From 21.6 million unique antihypertensive new users, we generate 6 076 775 effect size estimates for 699 872 research questions on 12 946 treatment comparisons. Through propensity score matching, we achieve balance on all baseline patient characteristics for 75% of estimates, observe 95.7% coverage in our effect-estimate 95% confidence intervals, find high between-database consistency, and achieve transitivity in 84.8% of triplet hypotheses. Compared with meta-analyses of RCTs, our results are consistent with 28 of 30 comparisons while providing narrower confidence intervals. Conclusion We find that these LEGEND results show high internal validity and are congruent with meta-analyses of RCTs. For these reasons we believe that evidence generated by LEGEND is of high quality and can inform medical decision-making where evidence is currently lacking. Subsequent publications will explore the clinical interpretations of this evidence.

Sodium glucose co-transporter 2 inhibitors (SGLT2i) are indicated for treatment of type 2 diabetes mellitus (T2DM); some SGLT2i have reported cardiovascular benefit, and some have reported risk of below-knee lower extremity (BKLE) amputation. This study examined the real-world comparative effectiveness within the SGLT2i class and compared with non-SGLT2i antihyperglycaemic agents.Data from 4 large US administrative claims databases were used to characterize risk and provide population-level estimates of canagliflozin's effects on hospitalization for heart failure (HHF) and BKLE amputation vs other SGLT2i and non-SGLT2i in T2DM patients. Comparative analyses using a propensity score-adjusted new-user cohort design examined relative hazards of outcomes across all new users and a subpopulation with established cardiovascular disease.Across the 4 databases (142 800 new users of canagliflozin, 110 897 new users of other SGLT2i, 460 885 new users of non-SGLT2i), the meta-analytic hazard ratio estimate for HHF with canagliflozin vs non-SGLT2i was 0.39 (95% CI, 0.26-0.60) in the on-treatment analysis. The estimate for BKLE amputation with canagliflozin vs non-SGLT2i was 0.75 (95% CI, 0.40-1.41) in the on-treatment analysis and 1.01 (95% CI, 0.93-1.10) in the intent-to-treat analysis. Effects in the subpopulation with established cardiovascular disease were similar for both outcomes. No consistent differences were observed between canagliflozin and other SGLT2i.In this large comprehensive analysis, canagliflozin and other SGLT2i demonstrated HHF benefits consistent with clinical trial data, but showed no increased risk of BKLE amputation vs non-SGLT2i. HHF and BKLE amputation results were similar in the subpopulation with established cardiovascular disease. This study helps further characterize the potential benefits and harms of SGLT2i in routine clinical practice to complement evidence from clinical trials and prior observational studies.

Objective To develop a conceptual prediction model framework containing standardized steps and describe the corresponding open-source software developed to consistently implement the framework across computational environments and observational healthcare databases to enable model sharing and reproducibility. Methods Based on existing best practices we propose a 5 step standardized framework for: (1) transparently defining the problem; (2) selecting suitable datasets; (3) constructing variables from the observational data; (4) learning the predictive model; and (5) validating the model performance. We implemented this framework as open-source software utilizing the Observational Medical Outcomes Partnership Common Data Model to enable convenient sharing of models and reproduction of model evaluation across multiple observational datasets. The software implementation contains default covariates and classifiers but the framework enables customization and extension. Results As a proof-of-concept, demonstrating the transparency and ease of model dissemination using the software, we developed prediction models for 21 different outcomes within a target population of people suffering from depression across 4 observational databases. All 84 models are available in an accessible online repository to be implemented by anyone with access to an observational database in the Common Data Model format. Conclusions The proof-of-concept study illustrates the framework’s ability to develop reproducible models that can be readily shared and offers the potential to perform extensive external validation of models, and improve their likelihood of clinical uptake. In future work the framework will be applied to perform an “all-by-all” prediction analysis to assess the observational data prediction domain across numerous target populations, outcomes and time, and risk settings.

Introduced in the 1950s, acetaminophen is one of the most widely used antipyretics and analgesics worldwide. In 1999, the International Agency for Research on Cancer (IARC) reviewed the epidemiologic studies of acetaminophen and the data were judged to be “inadequate” to conclude that it is carcinogenic. In 2019 the California Office of Environmental Health Hazard Assessment initiated a review process on the carcinogenic hazard potential of acetaminophen. To inform this review process, the authors performed a comprehensive literature search and identified 136 epidemiologic studies, which for most cancer types suggest no alteration in risk associated with acetaminophen use. For 3 cancer types, renal cell, liver, and some forms of lymphohematopoietic, some studies suggest an increased risk; however, multiple factors unique to acetaminophen need to be considered to determine if these results are real and clinically meaningful. The objective of this publication is to analyze the results of these epidemiologic studies using a framework that accounts for the inherent challenge of evaluating acetaminophen, including, broad population-wide use in multiple disease states, challenges with exposure measurement, protopathic bias, channeling bias, and recall bias. When evaluated using this framework, the data do not support a causal association between acetaminophen use and cancer.