Search

Electronic health record (EHR) or medical claims-based algorithms (i.e., operational definitions) can be used to define safety outcomes using real-world data. However, existing tools do not allow researchers and decision-makers to adequately appraise whether a particular algorithm is fit for purpose (FFP) to support regulatory decisions on drug safety surveillance. Our objective was to develop a tool to enable regulatory decision-makers and other stakeholders to appraise whether a given algorithm is FFP for a specific decision context.We drafted a set of 77 generic items informed by regulatory guidance documents, existing instruments, and publications. The outcome of ischemic stroke served as an exemplar to inform the development of draft items. The items were designed to be outcome independent. We conducted a three-round online Delphi panel to develop and refine the tool and achieve consensus on items ( 70% agreement) among panel participants composed of regulators, researchers from pharmaceutical organizations, academic clinicians, methodologists, pharmacoepidemiologists, and cardiologists. We conducted a qualitative analysis of panel responses. Five pairs of reviewers independently evaluated two ischemic stroke algorithm validation studies to test its application. We developed a user guide, with explanation and elaboration for each item, guidance on essential and additional elements for user responses, and an illustrative example of a complete assessment. Furthermore, we conducted a 2-h online stakeholder panel of 16 participants from regulatory agencies, academic institutions, and industry. We solicited input on key factors for an FFP assessment, their general reaction to the Algorithm CErtaInty Tool (ACE-IT), limitations of the tool, and its potential use.The expert panel reviewed and made changes to the initial list of 77 items. The panel achieved consensus on 38 items, and the final version of the ACE-IT includes 34 items after removal of duplicate items. Applying the tool to two ischemic stroke algorithms demonstrated challenges in its application and identified shared concepts addressed by more than one item. The ACE-IT was viewed positively by the majority of stakeholders. They identified that the tool could serve as an educational resource as well as an information-sharing platform. The time required to complete the assessment was identified as an important limitation. We consolidated items with shared concepts and added a preliminary screen section and a summary assessment box based on their input. The final version of the ACE-IT is a 34-item tool for assessing whether algorithm validation studies on safety outcomes are FFP. It comprises the domains of internal validity (24 items), external validity (seven items), and ethical conduct and reporting of the validation study (three items). The internal validity domain includes sections on objectives, data sources, population, outcomes, design and setting, statistical methods, reference standard, accuracy, and strengths and limitations. The external validity domain includes items that assess the generalizability to a proposed target study. The domain on ethics and transparency includes items on ethical conduct and reporting of the validation study.The ACE-IT supports a structured, transparent, and flexible approach for decision-makers to appraise whether electronic health record or medical claims-based algorithms for safety outcomes are FFP for a specific decision context. Reliability and validity testing using a larger sample of participants in other therapeutic areas and further modifications to reduce the time needed to complete the assessment are needed to fully evaluate its utility for regulatory decision-making.

Ursodeoxycholic acid (UDCA) remains the first-line therapy for primary biliary cholangitis (PBC); however, inadequate treatment response (ITR) is common. The UK-PBC Consortium developed the modified UDCA Response Score (m-URS) to predict ITR (using alkaline phosphatase [ALP] > 1.67 times the upper limit of normal [*ULN]) at 12 months post-UDCA initiation). Using data from the US-based Fibrotic Liver Disease Consortium, we assessed the m-URS in our multi-racial cohort. We then used a dynamic modeling approach to improve prediction accuracy. Using data collected at the time of UDCA initiation, we assessed the m-URS using the original formula; then, by calibrating coefficients to our data, we also assessed whether it remained accurate when using Paris II criteria for ITR. Next, we developed and validated a dynamic risk prediction model that included post-UDCA initiation laboratory data. Among 1578 patients (13% men; 8% African American, 9% Asian American/American Indian/Pacific Islander; 25% Hispanic), the rate of ITR was 27% using ALP > 1.67*ULN and 45% using Paris II criteria. M-URS accuracy was “very good” (AUROC = 0.87, sensitivity = 0.62, and specificity = 0.82) for ALP > 1.67*ULN and “moderate” (AUROC = 0.74, sensitivity = 0.57, and specificity = 0.70) for Paris II. Our dynamic model significantly improved accuracy for both definitions of ITR (ALP > 1.67*ULN: AUROC = 0.91; Paris II: AUROC = 0.81); specificity approached 100%. Roughly 9% of patients in our cohort were at the highest risk of ITR. Early identification of patients who will not respond to UDCA treatment using a dynamic prediction model based on longitudinal, repeated risk factor measurements may facilitate earlier introduction of adjuvant treatment.

The US Food and Drug Administration (FDA) created the Sentinel System in response to a requirement in the FDA Amendments Act of 2007 that the agency establish a system for monitoring risks associated with drug and biologic products using data from disparate sources. The Sentinel System has completed hundreds of analyses, including many that have directly informed regulatory decisions. The Sentinel System also was designed to support a national infrastructure for a learning health system. Sentinel governance and guiding principles were designed to facilitate Sentinel’s role as a national resource. The Sentinel System infrastructure now supports multiple non-FDA projects for stakeholders ranging from regulated industry to other federal agencies, international regulators, and academics. The Sentinel System is a working example of a learning health system that is expanding with the potential to create a global learning health system that can support medical product safety assessments and other research.

Narrative electronic prescribing instructions (NEPIs) are text that convey information on the administration or co-administration of a drug as directed by a prescriber. For researchers, NEPIs have the potential to advance our understanding of the risks and benefits of medications in populations; however, due to their unstructured nature, they are not often utilized. The goal of this scoping review was to evaluate how NEPIs are currently employed in research, identify opportunities and challenges for their broader application, and provide recommendations on their future use. The scoping review comprised a comprehensive literature review and a survey of key stakeholders. From the literature review, we identified 33 primary articles that described the use of NEPIs. The majority of articles (n = 19) identified issues with the quality of information in NEPIs compared with structured prescribing information; nine articles described the development of novel algorithms that performed well in extracting information from NEPIs, and five described the used of manual or simpler algorithms to extract prescribing information from NEPIs. A survey of 19 stakeholders indicated concerns for the quality of information in NEPIs and called for standardization of NEPIs to reduce data variability/errors. Nevertheless, stakeholders believed NEPIs present an opportunity to identify prescriber's intent for the prescription and to study temporal treatment patterns. In summary, NEPIs hold much promise for advancing the field of pharmacoepidemiology. Researchers should take advantage of addressing important questions that can be uniquely answered with NEPIs, but exercise caution when using this information and carefully consider the quality of the data.

Legislative and technological advancements over the past decade have given rise to the proliferation of healthcare data generated from routine clinical practice, often referred to as real-world data (RWD). These data have piqued the interest of healthcare stakeholders due to their potential utility in generating evidence to support clinical and regulatory decision making. In the oncology setting, studies leveraging RWD offer distinct advantages that are complementary to randomized controlled trials (RCTs). They also permit the conduct of investigations that may not be possible through prospective designs due to ethics or feasibility. Despite its promise, the use of RWD for the generation of clinical evidence remains controversial due to concerns of unmeasured confounding and other sources of bias that must be carefully addressed in the study design and analysis. To facilitate a better understanding of when RWD can provide reliable conclusions on drug effectiveness, we seek to conduct 10 RWD-based studies that emulate RCTs in oncology using a systematic, protocol-driven approach described herein. Results of this investigation will help inform clinical, scientific, and regulatory stakeholders on the applications of RWD in the context of product labeling expansion, drug safety, and comparative effectiveness in oncology.

IntroductionThis study aimed to assess data relevancy and data quality of the Innovation in Medical Evidence Development and Surveillance System Distributed Database (IMEDS-DD) for diabetes research and to evaluate comparability of its type 2 diabetes cohort to the general type 2 diabetes population.Research design and methodsA retrospective study was conducted using the IMEDS-DD. Eligible members were adults with a medical encounter between April 1, 2018 and March 31, 2019 (index period). Type 2 diabetes and co-existing conditions were determined using all data available from April 1, 2016 to the most recent encounter within the index period. Type 2 diabetes patient characteristics, comorbidities and hemoglobin A1c(HbA1c) values were summarized and compared with those reported in national benchmarks and literature.ResultsType 2 diabetes prevalence was 12.6% in the IMEDS-DD. Of 4 14 672 patients with type 2 diabetes, 52.8% were male, and the mean age was 65.0 (SD 13.3) years. Common comorbidities included hypertension (84.5%), hyperlipidemia (82.8%), obesity (45.3%), and cardiovascular disease (44.7%). Moderate-to-severe chronic kidney disease was observed in 20.2% patients. The most commonly used antihyperglycemic agents included metformin (35.7%), sulfonylureas (14.8%), and insulin (9.9%). Less than one-half (48.9%) had an HbA1cvalue recorded. These findings demonstrated the notable similarity in patient characteristics between type 2 diabetes populations identified within the IMEDS-DD and other large databases.ConclusionsDespite the limitations related to HbA1cdata, our findings indicate that the IMEDS-DD contains robust information on key data elements to conduct pharmacoepidemiological studies in diabetes, including member demographic and clinical characteristics and health services utilization.

Liver fibrosis is a common pathway of liver injury and is a feature of most chronic liver diseases. Fibrosis progression varies markedly in patients with hepatitis C virus (HCV). Liver stiffness has been recommended as a parameter of fibrosis progression/regression in patients with HCV.To investigate changes in liver stiffness measured by transient elastography (TE) in a large, racially diverse cohort of United States patients with chronic hepatitis C (CHC).We evaluated the differences in liver stiffness between patients treated with direct-acting antiviral (DAA) therapy and untreated patients. Patients had ≥ 2 TE measurements and no prior DAA exposure. We used linear regression to measure the change in liver stiffness between first and last TE in response to treatment, controlling for age, sex, race, diabetes, smoking status, human immunodeficiency virus status, baseline alanine aminotransferase, and baseline liver stiffness. Separate regression models analyzed the change in liver stiffness as measured by kPa, stratified by cirrhosis status.Of 813 patients, 419 (52%) initiated DAA treatment. Baseline liver stiffness was 12 kPa in 127 (16%). Median time between first and last TE was 11.7 and 12.7 mo among treated and untreated patients, respectively. There was no significant change in liver stiffness observed over time in either the group initiating DAA treatment (0.016 kPa/month; CI: -0.051, 0.084) or in the untreated group (0.001 kPa/mo; CI: -0.090, 0.092), controlling for covariates. A higher baseline kPa score was independently associated with decreased liver stiffness.DAA treatment was not associated with a differential change in liver stiffness over time in patients with CHC compared to untreated patients.

National policy initiatives1-3 and the advent of highly efficacious direct-acting antivirals4 set the stage to increase the identification and care of patients with hepatitis C virus (HCV). We implemented a multifaceted HCV care pathway, inclusive of automated screening alerts for all patients born between 1945 and 1965 as they are registered for appointments, reflex laboratory orders for positive HCV antibody results, and a care coordinator to facilitate diagnosis communication and engagement in follow-up care.5 We report the impact of that pathway on HCV screening, confirmation, diagnosis communication, and co-infection screening.