Search

Katy E Trinkley,1– 4 Michael G Kahn,5 Larry A Allen,2,4 Heather Haugen,6 Miranda E Kroehl,7 Chen-Tan Lin,2,3 Daniel C Malone,8 Daniel D Matlock2,4,9 1Department of Clinical Pharmacy, University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, CO, USA; 2Department of Medicine, University of Colorado School of Medicine, Aurora, CO, USA; 3Clinical Informatics, University of Colorado Health, Aurora, CO, USA; 4Adult and Child Consortium for Outcomes Research and Delivery Science, Aurora, CO, USA; 5Section of Informatics and Data Science, Department of Pediatrics, University of Colorado School of Medicine, Aurora, CO, USA; 6University of Colorado, Colorado Clinical and Translational Sciences Institute (CCTSI), Aurora, CO, USA; 7Charter Communications Corporation, Greenwood Village, CO, USA; 8Department of Pharmacotherapy, University of Utah Skaggs College of Pharmacy, Salt Lake City, UT, USA; 9VA Eastern Colorado Geriatric Research Education and Clinical Center, Aurora, CO, USACorrespondence: Katy E TrinkleyUniversity of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences, 12850 E Montview Blvd., Mail Stop C238, Aurora, CO 80045, USATel +1-303-724-6563Fax +1-303-724-0979Email katy.trinkley@cuanschutz.eduIntroduction: Consideration of patient preferences for guideline-directed medical therapies (GDMT) for heart failure with reduced ejection fraction (HFrEF) may help improve major gaps in prescribing and adherence. This study aimed to identify the range and relative priority of factors influencing patients’ decisions to take HFrEF medications.Materials and Methods: This was a convergent mixed methods study of patients with HFrEF. Focus groups were conducted to identify a list of factors followed by individuals rating and ranking the influence of each factor on their decision to take a medication. Using thematic analysis, we summarized preferences into categories.Results: Two focus groups with 13 participants reported 22 factors. Of the factors, “keeping you alive” was most commonly ranked in the top three (seven participants), followed by “communication and understanding” (six participants). Factors were summarized into six categories (listed in order of patient-reported influence): 1) demonstrated improvements in quality of life and longevity, 2) decreased risk of hospitalization, 3) opportunity for shared decision-making and trust in provider, 4) absence of adverse events, 5) affordability, and 6) convenience of taking and absence of interference with daily life.Conclusion: Patients prioritize treatment benefits and being informed more than risks, cost and inconvenience of taking HFrEF medications.Keywords: heart failure, treatment preferences, medication preferences, patient preferences

Clinical research informatics is the rapidly evolving sub-discipline within biomedical informatics that focuses on developing new informatics theories, tools, and solutions to accelerate the full translational continuum: basic research to clinical trials (T1), clinical trials to academic health center practice (T2), diffusion and implementation to community practice (T3), and 'real world' outcomes (T4). We present a conceptual model based on an informatics-enabled clinical research workflow, integration across heterogeneous data sources, and core informatics tools and platforms. We use this conceptual model to highlight 18 new articles in the JAMIA special issue on clinical research informatics. [ABSTRACT FROM AUTHOR]

Objectives: Accurate record linkage (RL) enables consolidation and de-duplication of data from disparate datasets, resulting in more comprehensive and complete patient data. However, conducting RL with low quality or unfit data can waste institutional resources on poor linkage results. We aim to evaluate data linkability to enhance the effectiveness of record linkage., Materials and Methods: We describe a systematic approach using data fitness ("linkability") measures, defined as metrics that characterize the availability, discriminatory power, and distribution of potential variables for RL. We used the isolation forest algorithm to detect abnormal linkability values from 188 sites in Indiana and Colorado, and manually reviewed the data to understand the cause of anomalies., Result: We calculated 10 linkability metrics for 11 potential linkage variables (LVs) across 188 sites for a total of 20 680 linkability metrics. Potential LVs such as first name, last name, date of birth, and sex have low missing data rates, while Social Security Number vary widely in completeness among all sites. We investigated anomalous linkability values to identify the cause of many records having identical values in certain LVs, issues with placeholder values disguising data missingness, and orphan records., Discussion: The fitness of a variable for RL is determined by its availability and its discriminatory power to uniquely identify individuals. These results highlight the need for awareness of placeholder values, which inform the selection of variables and methods to optimize RL performance., Conclusion: Evaluating linkability measures using the isolation forest algorithm to highlight anomalous findings can help identify fitness-for-use issues that must be addressed before initiating the RL process to ensure high-quality linkage outcomes., (© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Objectives: The Multi-State EHR-Based Network for Disease Surveillance (MENDS) is a population-based chronic disease surveillance distributed data network that uses institution-specific extraction-transformation-load (ETL) routines. MENDS-on-FHIR examined using Health Language Seven's Fast Healthcare Interoperability Resources (HL7 ® FHIR ® ) and US Core Implementation Guide (US Core IG) compliant resources derived from the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) to create a standards-based ETL pipeline., Materials and Methods: The input data source was a research data warehouse containing clinical and administrative data in OMOP CDM Version 5.3 format. OMOP-to-FHIR transformations, using a unique JavaScript Object Notation (JSON)-to-JSON transformation language called Whistle, created FHIR R4 V4.0.1/US Core IG V4.0.0 conformant resources that were stored in a local FHIR server. A REST-based Bulk FHIR $export request extracted FHIR resources to populate a local MENDS database., Results: Eleven OMOP tables were used to create 10 FHIR/US Core compliant resource types. A total of 1.13 trillion resources were extracted and inserted into the MENDS repository. A very low rate of non-compliant resources was observed., Discussion: OMOP-to-FHIR transformation results passed validation with less than a 1% non-compliance rate. These standards-compliant FHIR resources provided standardized data elements required by the MENDS surveillance use case. The Bulk FHIR application programming interface (API) enabled population-level data exchange using interoperable FHIR resources. The OMOP-to-FHIR transformation pipeline creates a FHIR interface for accessing OMOP data., Conclusion: MENDS-on-FHIR successfully replaced custom ETL with standards-based interoperable FHIR resources using Bulk FHIR. The OMOP-to-FHIR transformations provide an alternative mechanism for sharing OMOP data., Competing Interests: B.Z. and J.A. are affiliated with an organization that has funding from the Massachusetts Department of Public Health for support and development of Electronic Medical Record Support for Public Health (ESP) and MDPHnet, which is the underlying technology of MENDS. All other authors declare no competing interests. No copyrighted materials were used in this article., (© The Author(s) 2024. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Translational research requires data at multiple scales of biological organization. Advancements in sequencing and multi-omics technologies have increased the availability of these data, but researchers face significant integration challenges. Knowledge graphs (KGs) are used to model complex phenomena, and methods exist to construct them automatically. However, tackling complex biomedical integration problems requires flexibility in the way knowledge is modeled. Moreover, existing KG construction methods provide robust tooling at the cost of fixed or limited choices among knowledge representation models. PheKnowLator (Phenotype Knowledge Translator) is a semantic ecosystem for automating the FAIR (Findable, Accessible, Interoperable, and Reusable) construction of ontologically grounded KGs with fully customizable knowledge representation. The ecosystem includes KG construction resources (e.g., data preparation APIs), analysis tools (e.g., SPARQL endpoint resources and abstraction algorithms), and benchmarks (e.g., prebuilt KGs). We evaluated the ecosystem by systematically comparing it to existing open-source KG construction methods and by analyzing its computational performance when used to construct 12 different large-scale KGs. With flexible knowledge representation, PheKnowLator enables fully customizable KGs without compromising performance or usability., (© 2024. The Author(s).)

Precision medicine initiatives across the globe have led to a revolution of repositories linking large-scale genomic data with electronic health records, enabling genomic analyses across the entire phenome. Many of these initiatives focus solely on research insights, leading to limited direct benefit to patients. We describe the biobank at the Colorado Center for Personalized Medicine (CCPM Biobank) that was jointly developed by the University of Colorado Anschutz Medical Campus and UCHealth to serve as a unique, dual-purpose research and clinical resource accelerating personalized medicine. This living resource currently has more than 200,000 participants with ongoing recruitment. We highlight the clinical, laboratory, regulatory, and HIPAA-compliant informatics infrastructure along with our stakeholder engagement, consent, recontact, and participant engagement strategies. We characterize aspects of genetic and geographic diversity unique to the Rocky Mountain region, the primary catchment area for CCPM Biobank participants. We leverage linked health and demographic information of the CCPM Biobank participant population to demonstrate the utility of the CCPM Biobank to replicate complex trait associations in the first 33,674 genotyped individuals across multiple disease domains. Finally, we describe our current efforts toward return of clinical genetic test results, including high-impact pathogenic variants and pharmacogenetic information, and our broader goals as the CCPM Biobank continues to grow. Bringing clinical and research interests together fosters unique clinical and translational questions that can be addressed from the large EHR-linked CCPM Biobank resource within a HIPAA- and CLIA-certified environment., Competing Interests: Declaration of interests K.C.B. owns stock in Tempus and Galatea Bio and is an employee of Oxford Nanopore Technologies. C.R.G. owns stock in 23andMe, Inc., (Copyright © 2023 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: In a randomized controlled trial, we found that applying implementation science (IS) methods and best practices in clinical decision support (CDS) design to create a locally customized, "enhanced" CDS significantly improved evidence-based prescribing of β blockers (BB) for heart failure compared with an unmodified commercially available CDS. At trial conclusion, the enhanced CDS was expanded to all sites. The purpose of this study was to evaluate the real-world sustained effect of the enhanced CDS compared with the commercial CDS., Methods: In this natural experiment of 28 primary care clinics, we compared clinics exposed to the commercial CDS (preperiod) to clinics exposed to the enhanced CDS (both periods). The primary effectiveness outcome was the proportion of alerts resulting in a BB prescription. Secondary outcomes included patient reach and clinician adoption (dismissals)., Results: There were 367 alerts for 183 unique patients and 171 unique clinicians (pre: March 2019-August 2019; post: October 2019-March 2020). The enhanced CDS increased prescribing by 26.1% compared with the commercial (95% confidence interval [CI]: 17.0-35.1%), which is consistent with the 24% increase in the previous study. The odds of adopting the enhanced CDS was 81% compared with 29% with the commercial (odds ratio: 4.17, 95% CI: 1.96-8.85). The enhanced CDS adoption and effectiveness rates were 62 and 14% in the preperiod and 92 and 10% in the postperiod., Conclusion: Applying IS methods with CDS best practices was associated with improved and sustained clinician adoption and effectiveness compared with a commercially available CDS tool., Competing Interests: None declared., (Thieme. All rights reserved.)

Common data models solve many challenges of standardizing electronic health record (EHR) data but are unable to semantically integrate all of the resources needed for deep phenotyping. Open Biological and Biomedical Ontology (OBO) Foundry ontologies provide computable representations of biological knowledge and enable the integration of heterogeneous data. However, mapping EHR data to OBO ontologies requires significant manual curation and domain expertise. We introduce OMOP2OBO, an algorithm for mapping Observational Medical Outcomes Partnership (OMOP) vocabularies to OBO ontologies. Using OMOP2OBO, we produced mappings for 92,367 conditions, 8611 drug ingredients, and 10,673 measurement results, which covered 68-99% of concepts used in clinical practice when examined across 24 hospitals. When used to phenotype rare disease patients, the mappings helped systematically identify undiagnosed patients who might benefit from genetic testing. By aligning OMOP vocabularies to OBO ontologies our algorithm presents new opportunities to advance EHR-based deep phenotyping., (© 2023. The Author(s).)

Patient representation learning methods create rich representations of complex data and have potential to further advance the development of computational phenotypes (CP). Currently, these methods are either applied to small predefined concept sets or all available patient data, limiting the potential for novel discovery and reducing the explainability of the resulting representations. We report on an extensive, data-driven characterization of the utility of patient representation learning methods for the purpose of CP development or automatization. We conducted ablation studies to examine the impact of patient representations, built using data from different combinations of data types and sampling windows on rare disease classification. We demonstrated that the data type and sampling window directly impact classification and clustering performance, and these results differ by rare disease group. Our results, although preliminary, exemplify the importance of and need for data-driven characterization in patient representation-based CP development pipelines., (©2022 AMIA - All rights reserved.)

Background: Adverse events of special interest (AESIs) were pre-specified to be monitored for the COVID-19 vaccines. Some AESIs are not only associated with the vaccines, but with COVID-19. Our aim was to characterise the incidence rates of AESIs following SARS-CoV-2 infection in patients and compare these to historical rates in the general population., Methods: A multi-national cohort study with data from primary care, electronic health records, and insurance claims mapped to a common data model. This study's evidence was collected between Jan 1, 2017 and the conclusion of each database (which ranged from Jul 2020 to May 2022). The 16 pre-specified prevalent AESIs were: acute myocardial infarction, anaphylaxis, appendicitis, Bell's palsy, deep vein thrombosis, disseminated intravascular coagulation, encephalomyelitis, Guillain- Barré syndrome, haemorrhagic stroke, non-haemorrhagic stroke, immune thrombocytopenia, myocarditis/pericarditis, narcolepsy, pulmonary embolism, transverse myelitis, and thrombosis with thrombocytopenia. Age-sex standardised incidence rate ratios (SIR) were estimated to compare post-COVID-19 to pre-pandemic rates in each of the databases., Findings: Substantial heterogeneity by age was seen for AESI rates, with some clearly increasing with age but others following the opposite trend. Similarly, differences were also observed across databases for same health outcome and age-sex strata. All studied AESIs appeared consistently more common in the post-COVID-19 compared to the historical cohorts, with related meta-analytic SIRs ranging from 1.32 (1.05 to 1.66) for narcolepsy to 11.70 (10.10 to 13.70) for pulmonary embolism., Interpretation: Our findings suggest all AESIs are more common after COVID-19 than in the general population. Thromboembolic events were particularly common, and over 10-fold more so. More research is needed to contextualise post-COVID-19 complications in the longer term., Funding: None., Competing Interests: EAV, AS, CB, RM, KH, AGS, GR, MJS, and PBR are employees of Janssen Research and Development LLC and shareholders of Johnson & Johnson (J&J) stock. Many data partners (including JMR, JMRA, MAM, MdW, LHJ, GB, PRR) on this project have received funding from the Innovative Medicines Initiative (IMI) 2 Joint Undertaking (JU) under grant agreement No 806968. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. The funding source had no role in writing the manuscript or the decision to submit it for publication. Odysseus Data Services, where SvS is employed, is a beneficiary of the mentioned IMI 2 JU project. DPA has received support from the EMA and Innovative Medicines Initiative. DPA's research group has received grant support from Amgen, Chesi-Taylor, Novartis, GSK, UCB Biopharma and J&J. DPA's department has received advisory/consultancy fees from AstraZeneca, Amgen, Astellas, J&J, and UCB Biopharma. DPA's department has received fees for speaker services from Amgen and UCB Biopharma. DPA's has other financial or non-financial interests in Janssen, on behalf of the IMI-funded EHDEN and European Medical Information Framework (EMIF) consortiums, and Synapse Management Partners have supported training programmes organised by DPA's department that was open for external participants. TDS received financial support from the Institute de Salud Carlos Ill (ISCIII; Miguel Servet 2021: CP21/00023) which is co-funded by the European Union. The funding source had no role in writing the manuscript or the decision to submit it for publication. PRR, MdW, LHJ work for a research group who received/receives unconditional research grants from Chiesi, Amgen, UCB, J&J, European Medicines Agency (EMA), and IMI. GH provided funding for use of the CUIMC database and the OHDSI infrastructure. GH received support through a grant to Columbia University (NIH R01 LM006910). MAS receives grants from the US Department of Veterans Affairs within the scope of this work, and grants and contracts from the US National Institutes of Health, US Food & Drug Administration, and Janssen Research & Development outside the scope of this work. EPM receives salary support from the Department of Medicine at University of Calgary. EPM receives consulting fees from Janssen Research & Development for work that is outside the scope of the current study. FM's main employment is for Heliant d.o.o, an EHR provider in Serbia. JDP received salary support through the Universidad de Norte. DJD and ZC are full-time employees of CPRD, which provides contract research services for government and related healthcare authorities, and the pharmaceutical industry, including Janssen. FN holds some AstraZeneca shares and was employed there until 2019. RV, AJB, AP2, LD, and CH received partial funding for this work by the University of California Health System and its Center for Data-Driven Insights and Innovation. MGK received support for the manuscript via institutional (internal funding) only. AJB's research has been funded by NIH, Peraton (as the prime on an NIH contract), Genentech, Johnson and Johnson, FDA, Robert Wood Johnson Foundation, Leon Lowenstein Foundation, Intervalien Foundation, Priscilla Chan and Mark Zuckerberg, the Barbara and Gerson Bakar Foundation, and in the recent past, the March of Dimes, Juvenile Diabetes Research Foundation, California Governor’s Office of Planning and Research, California Institute for Regenerative Medicine, L’Oréal, and Progenity. AJB receives royalty payments through Stanford University, for several patents and other disclosures licensed to NuMedii and Personalis. AJB is a co-founder and consultant to Personalis and NuMedii; consultant to Mango Tree Corporation, and in the recent past, Samsung, 10x Genomics, Helix, Pathway Genomics, and Verinata (Illumina); has served on paid advisory panels or boards for Geisinger Health, Regenstrief Institute, Gerson Lehman Group, AlphaSights, Covance, Novartis, Genentech, Merck, and Roche. AJB has received honoraria and travel reimbursement for invited talks from J&J, Roche, Genentech, Pfizer, Merck, Lilly, Takeda, Varian, Mars, Siemens, Optum, Abbott, Celgene, AstraZeneca, AbbVie, Westat, many academic institutions, medical or disease specific foundations and associations, and health systems. AJB is a shareholder in Personalis and NuMedii; is a minor shareholder in Apple, Meta (Facebook), Alphabet (Google), Microsoft, Amazon, Snap, 10x Genomics, Illumina, Regeneron, Sanofi, Pfizer, Royalty Pharma, Moderna, Sutro, Doximity, BioNtech, Invitae, Pacific Biosciences, Editas Medicine, Nuna Health, Assay Depot, Vet24seven, and several other non-health related companies and mutual funds. DRM has grants or contracts from Wellcome Trust Clinical Research Fellowship (214588/Z/18/Z). FB has a leadership or fiduciary role as the program chair of j-ISBA., (© 2023 The Author(s).)

Objectives: Manual record review is a crucial step for electronic health record (EHR)-based research, but it has poor workflows and is error prone. We sought to build a tool that provides a unified environment for data review and chart abstraction data entry., Materials and Methods: ReviewR is an open-source R Shiny application that can be deployed on a single machine or made available to multiple users. It supports multiple data models and database systems, and integrates with the REDCap API for storing abstraction results., Results: We describe 2 real-world uses and extensions of ReviewR. Since its release in April 2021 as a package on CRAN it has been downloaded 2204 times., Discussion and Conclusion: ReviewR provides an easily accessible review interface for clinical data warehouses. Its modular, extensible, and open source nature afford future expansion by other researchers., (© The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Background: Post-acute sequelae of SARS-CoV-2 infection, known as long COVID, have severely affected recovery from the COVID-19 pandemic for patients and society alike. Long COVID is characterised by evolving, heterogeneous symptoms, making it challenging to derive an unambiguous definition. Studies of electronic health records are a crucial element of the US National Institutes of Health's RECOVER Initiative, which is addressing the urgent need to understand long COVID, identify treatments, and accurately identify who has it-the latter is the aim of this study., Methods: Using the National COVID Cohort Collaborative's (N3C) electronic health record repository, we developed XGBoost machine learning models to identify potential patients with long COVID. We defined our base population (n=1 793 604) as any non-deceased adult patient (age ≥18 years) with either an International Classification of Diseases-10-Clinical Modification COVID-19 diagnosis code (U07.1) from an inpatient or emergency visit, or a positive SARS-CoV-2 PCR or antigen test, and for whom at least 90 days have passed since COVID-19 index date. We examined demographics, health-care utilisation, diagnoses, and medications for 97 995 adults with COVID-19. We used data on these features and 597 patients from a long COVID clinic to train three machine learning models to identify potential long COVID among all patients with COVID-19, patients hospitalised with COVID-19, and patients who had COVID-19 but were not hospitalised. Feature importance was determined via Shapley values. We further validated the models on data from a fourth site., Findings: Our models identified, with high accuracy, patients who potentially have long COVID, achieving areas under the receiver operator characteristic curve of 0·92 (all patients), 0·90 (hospitalised), and 0·85 (non-hospitalised). Important features, as defined by Shapley values, include rate of health-care utilisation, patient age, dyspnoea, and other diagnosis and medication information available within the electronic health record., Interpretation: Patients identified by our models as potentially having long COVID can be interpreted as patients warranting care at a specialty clinic for long COVID, which is an essential proxy for long COVID diagnosis as its definition continues to evolve. We also achieve the urgent goal of identifying potential long COVID in patients for clinical trials. As more data sources are identified, our models can be retrained and tuned based on the needs of individual studies., Funding: US National Institutes of Health and National Center for Advancing Translational Sciences through the RECOVER Initiative., Competing Interests: Declaration of interests ATG is an employee of Palantir Technologies. ERP, JPD, SEJ, RRD, CGC, TDB, JAM, RM, AW, and MAH report research funding from the NIH. ERP and MGK report research funding from PCORI. MAH and JAM are co-founders of Pryzm Health. All other authors declare no competing interests., (Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 license. Published by Elsevier Ltd.. All rights reserved.)

Objective: Clinical research data warehouses (RDWs) linked to genomic pipelines and open data archives are being created to support innovative, complex data-driven discoveries. The computing and storage needs of these research environments may quickly exceed the capacity of on-premises systems. New RDWs are migrating to cloud platforms for the scalability and flexibility needed to meet these challenges. We describe our experience in migrating a multi-institutional RDW to a public cloud., Materials and Methods: This study is descriptive. Primary materials included internal and public presentations before and after the transition, analysis documents, and actual billing records. Findings were aggregated into topical categories., Results: Eight categories of migration issues were identified. Unanticipated challenges included legacy system limitations; network, computing, and storage architectures that realize performance and cost benefits in the face of hyper-innovation, complex security reviews and approvals, and limited cloud consulting expertise., Discussion: Cloud architectures enable previously unavailable capabilities, but numerous pitfalls can impede realizing the full benefits of a cloud environment. Rapid changes in cloud capabilities can quickly obsolete existing architectures and associated institutional policies. Touchpoints with on-premise networks and systems can add unforeseen complexity. Governance, resource management, and cost oversight are critical to allow rapid innovation while minimizing wasted resources and unnecessary costs., Conclusions: Migrating our RDW to the cloud has enabled capabilities and innovations that would not have been possible with an on-premises environment. Notwithstanding the challenges of managing cloud resources, the resulting RDW capabilities have been highly positive to our institution, research community, and partners., (© The Author(s) 2021. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Background: There is a growing interest in using person-generated wearable device data for biomedical research, but there are also concerns regarding the quality of data such as missing or incorrect data. This emphasizes the importance of assessing data quality before conducting research. In order to perform data quality assessments, it is essential to define what data quality means for person-generated wearable device data by identifying the data quality dimensions., Objective: This study aims to identify data quality dimensions for person-generated wearable device data for research purposes., Methods: This study was conducted in 3 phases: literature review, survey, and focus group discussion. The literature review was conducted following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guideline to identify factors affecting data quality and its associated data quality challenges. In addition, we conducted a survey to confirm and complement results from the literature review and to understand researchers' perceptions on data quality dimensions that were previously identified as dimensions for the secondary use of electronic health record (EHR) data. We sent the survey to researchers with experience in analyzing wearable device data. Focus group discussion sessions were conducted with domain experts to derive data quality dimensions for person-generated wearable device data. On the basis of the results from the literature review and survey, a facilitator proposed potential data quality dimensions relevant to person-generated wearable device data, and the domain experts accepted or rejected the suggested dimensions., Results: In total, 19 studies were included in the literature review, and 3 major themes emerged: device- and technical-related, user-related, and data governance-related factors. The associated data quality problems were incomplete data, incorrect data, and heterogeneous data. A total of 20 respondents answered the survey. The major data quality challenges faced by researchers were completeness, accuracy, and plausibility. The importance ratings on data quality dimensions in an existing framework showed that the dimensions for secondary use of EHR data are applicable to person-generated wearable device data. There were 3 focus group sessions with domain experts in data quality and wearable device research. The experts concluded that intrinsic data quality features, such as conformance, completeness, and plausibility, and contextual and fitness-for-use data quality features, such as completeness (breadth and density) and temporal data granularity, are important data quality dimensions for assessing person-generated wearable device data for research purposes., Conclusions: In this study, intrinsic and contextual and fitness-for-use data quality dimensions for person-generated wearable device data were identified. The dimensions were adapted from data quality terminologies and frameworks for the secondary use of EHR data with a few modifications. Further research on how data quality can be assessed with respect to each dimension is needed., (©Sylvia Cho, Chunhua Weng, Michael G Kahn, Karthik Natarajan. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 23.12.2021.)

Introduction: Learning health systems can help estimate chronic disease prevalence through distributed data networks (DDNs). Concerns remain about bias introduced to DDN prevalence estimates when individuals seeking care across systems are counted multiple times. This paper describes a process to deduplicate individuals for DDN prevalence estimates., Methods: We operationalized a two-step deduplication process, leveraging health information exchange (HIE)-assigned network identifiers, within the Colorado Health Observation Regional Data Service (CHORDS) DDN. We generated prevalence estimates for type 1 and type 2 diabetes among pediatric patients (0-17 years) with at least one 2017 encounter in one of two geographically-proximate DDN partners. We assessed the extent of cross-system duplication and its effect on prevalence estimates., Results: We identified 218 437 unique pediatric patients seen across systems during 2017, including 7628 (3.5%) seen in both. We found no measurable difference in prevalence after deduplication. The number of cases we identified differed slightly by data reconciliation strategy. Concordance of linked patients' demographic attributes varied by attribute., Conclusions: We implemented an HIE-dependent, extensible process that deduplicates individuals for less biased prevalence estimates in a DDN. Our null pilot findings have limited generalizability. Overlap was small and likely insufficient to influence prevalence estimates. Other factors, including the number and size of partners, the matching algorithm, and the electronic phenotype may influence the degree of deduplication bias. Additional use cases may help improve understanding of duplication bias and reveal other principles and insights. This study informed how DDNs could support learning health systems' response to public health challenges and improve regional health., Competing Interests: The authors have no conflicts of interest to declare., (© 2021 The Authors. Learning Health Systems published by Wiley Periodicals LLC on behalf of University of Michigan.)

Background: Since the onset of the SARS-CoV-2 pandemic, most clinical testing has focused on RT-PCR 1 . Host epigenome manipulation post coronavirus infection 2-4 suggests that DNA methylation signatures may differentiate patients with SARS-CoV-2 infection from uninfected individuals, and help predict COVID-19 disease severity, even at initial presentation., Methods: We customized Illumina's Infinium MethylationEPIC array to enhance immune response detection and profiled peripheral blood samples from 164 COVID-19 patients with longitudinal measurements of disease severity and 296 patient controls., Results: Epigenome-wide association analysis revealed 13,033 genome-wide significant methylation sites for case-vs-control status. Genes and pathways involved in interferon signaling and viral response were significantly enriched among differentially methylated sites. We observe highly significant associations at genes previously reported in genetic association studies (e.g. IRF7, OAS1). Using machine learning techniques, models built using sparse regression yielded highly predictive findings: cross-validated best fit AUC was 93.6% for case-vs-control status, and 79.1%, 80.8%, and 84.4% for hospitalization, ICU admission, and progression to death, respectively., Conclusions: In summary, the strong COVID-19-specific epigenetic signature in peripheral blood driven by key immune-related pathways related to infection status, disease severity, and clinical deterioration provides insights useful for diagnosis and prognosis of patients with viral infections., (© 2021. The Author(s).)

Objective: Data quality (DQ) must be consistently defined in context. The attributes, metadata, and context of longitudinal real-world data (RWD) have not been formalized for quality improvement across the data production and curation life cycle. We sought to complete a literature review on DQ assessment frameworks, indicators and tools for research, public health, service, and quality improvement across the data life cycle., Materials and Methods: The review followed PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Databases from health, physical and social sciences were used: Cinahl, Embase, Scopus, ProQuest, Emcare, PsycINFO, Compendex, and Inspec. Embase was used instead of PubMed (an interface to search MEDLINE) because it includes all MeSH (Medical Subject Headings) terms used and journals in MEDLINE as well as additional unique journals and conference abstracts. A combined data life cycle and quality framework guided the search of published and gray literature for DQ frameworks, indicators, and tools. At least 2 authors independently identified articles for inclusion and extracted and categorized DQ concepts and constructs. All authors discussed findings iteratively until consensus was reached., Results: The 120 included articles yielded concepts related to contextual (data source, custodian, and user) and technical (interoperability) factors across the data life cycle. Contextual DQ subcategories included relevance, usability, accessibility, timeliness, and trust. Well-tested computable DQ indicators and assessment tools were also found., Conclusions: A DQ assessment framework that covers intrinsic, technical, and contextual categories across the data life cycle enables assessment and management of RWD repositories to ensure fitness for purpose. Balancing security, privacy, and FAIR principles requires trust and reciprocity, transparent governance, and organizational cultures that value good documentation., (© The Author(s) 2021. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

One of the challenges of teaching applied data science courses is managing individual students' local computing environment. This is especially challenging when teaching massively open online courses (MOOCs) where students come from across the globe and have a variety of access to and types of computing systems. There are additional challenges with using sensitive health information for clinical data science education. Here we describe the development and performance of a computing platform developed to support a series of MOOCs in clinical data science. This platform was designed to restrict and log all access to health datasets while also being scalable, accessible, secure, privacy preserving, and easy to access. Over the 19 months the platform has been live it has supported the computation of more than 2300 students from 101 countries., (©2021 AMIA - All rights reserved.)

Background: Limited consideration of clinical decision support (CDS) design best practices, such as a user-centered design, is often cited as a key barrier to CDS adoption and effectiveness. The application of CDS best practices is resource intensive; thus, institutions often rely on commercially available CDS tools that are created to meet the generalized needs of many institutions and are not user centered. Beyond resource availability, insufficient guidance on how to address key aspects of implementation, such as contextual factors, may also limit the application of CDS best practices. An implementation science (IS) framework could provide needed guidance and increase the reproducibility of CDS implementations., Objective: This study aims to compare the effectiveness of an enhanced CDS tool informed by CDS best practices and an IS framework with a generic, commercially available CDS tool., Methods: We conducted an explanatory sequential mixed methods study. An IS-enhanced and commercial CDS alert were compared in a cluster randomized trial across 28 primary care clinics. Both alerts aimed to improve beta-blocker prescribing for heart failure. The enhanced alert was informed by CDS best practices and the Practical, Robust, Implementation, and Sustainability Model (PRISM) IS framework, whereas the commercial alert followed vendor-supplied specifications. Following PRISM, the enhanced alert was informed by iterative, multilevel stakeholder input and the dynamic interactions of the internal and external environment. Outcomes aligned with PRISM's evaluation measures, including patient reach, clinician adoption, and changes in prescribing behavior. Clinicians exposed to each alert were interviewed to identify design features that might influence adoption. The interviews were analyzed using a thematic approach., Results: Between March 15 and August 23, 2019, the enhanced alert fired for 61 patients (106 alerts, 87 clinicians) and the commercial alert fired for 26 patients (59 alerts, 31 clinicians). The adoption and effectiveness of the enhanced alert were significantly higher than those of the commercial alert (62% vs 29% alerts adopted, P<.001; 14% vs 0% changed prescribing, P=.006). Of the 21 clinicians interviewed, most stated that they preferred the enhanced alert., Conclusions: The results of this study suggest that applying CDS best practices with an IS framework to create CDS tools improves implementation success compared with a commercially available tool., Trial Registration: ClinicalTrials.gov NCT04028557; http://clinicaltrials.gov/ct2/show/NCT04028557., (©Katy E Trinkley, Miranda E Kroehl, Michael G Kahn, Larry A Allen, Tellen D Bennett, Gary Hale, Heather Haugen, Simeon Heckman, David P Kao, Janet Kim, Daniel M Matlock, Daniel C Malone, Robert L Page 2nd, Jessica Stine, Krithika Suresh, Lauren Wells, Chen-Tan Lin. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 22.03.2021.)

Background: There is increasing interest in reusing person-generated wearable device data for research purposes, which raises concerns about data quality. However, the amount of literature on data quality challenges, specifically those for person-generated wearable device data, is sparse., Objective: This study aims to systematically review the literature on factors affecting the quality of person-generated wearable device data and their associated intrinsic data quality challenges for research., Methods: The literature was searched in the PubMed, Association for Computing Machinery, Institute of Electrical and Electronics Engineers, and Google Scholar databases by using search terms related to wearable devices and data quality. By using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, studies were reviewed to identify factors affecting the quality of wearable device data. Studies were eligible if they included content on the data quality of wearable devices, such as fitness trackers and sleep monitors. Both research-grade and consumer-grade wearable devices were included in the review. Relevant content was annotated and iteratively categorized into semantically similar factors until a consensus was reached. If any data quality challenges were mentioned in the study, those contents were extracted and categorized as well., Results: A total of 19 papers were included in this review. We identified three high-level factors that affect data quality-device- and technical-related factors, user-related factors, and data governance-related factors. Device- and technical-related factors include problems with hardware, software, and the connectivity of the device; user-related factors include device nonwear and user error; and data governance-related factors include a lack of standardization. The identified factors can potentially lead to intrinsic data quality challenges, such as incomplete, incorrect, and heterogeneous data. Although missing and incorrect data are widely known data quality challenges for wearable devices, the heterogeneity of data is another aspect of data quality that should be considered for wearable devices. Heterogeneity in wearable device data exists at three levels: heterogeneity in data generated by a single person using a single device (within-person heterogeneity); heterogeneity in data generated by multiple people who use the same brand, model, and version of a device (between-person heterogeneity); and heterogeneity in data generated from multiple people using different devices (between-person heterogeneity), which would apply especially to data collected under a bring-your-own-device policy., Conclusions: Our study identifies potential intrinsic data quality challenges that could occur when analyzing wearable device data for research and three major contributing factors for these challenges. As poor data quality can compromise the reliability and accuracy of research results, further investigation is needed on how to address the data quality challenges of wearable devices., (©Sylvia Cho, Ipek Ensari, Chunhua Weng, Michael G Kahn, Karthik Natarajan. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 19.03.2021.)

Objective: Incidence of youth-onset diabetes in India has not been well described. Comparison of incidence, across diabetes registries, has the potential to inform hypotheses for risk factors. We sought to compare the incidence of diabetes in the U.S.-based registry of youth onset diabetes (SEARCH) to the Registry of Diabetes with Young Age at Onset (YDR-Chennai and New Delhi regions) in India., Methods: We harmonized data from both SEARCH and YDR to the Observational Medical Outcomes Partnership (OMOP) Common Data Model. Data were from youth registered with incident diabetes (2006-2012). Denominators were from census and membership data. We calculated diabetes incidence by averaging the total cases across the entire follow-up period and dividing this by the estimated census population corresponding to the source population for case ascertainment. Incidence was calculated for each of the registries and compared by type and within age and sex categories using a 2-sided, skew-corrected inverted score test., Results: Incidence of type 1 was higher in SEARCH (21.2 cases/100 000 [95% CI: 19.9, 22.5]) than YDR (4.9 cases/100 000 [95% CI: 4.3, 5.6]). Incidence of type 2 diabetes was also higher in SEARCH (5.9 cases/100 000 [95% CI: 5.3, 6.6] in SEARCH vs 0.5/cases/100 000 [95% CI: 0.3, 0.7] in YDR). The age distribution of incident type 1 diabetes cases was similar across registries, whereas type 2 diabetes incidence was higher at an earlier age in SEARCH. Sex differences existed in SEARCH only, with a higher rate of type 2 diabetes among females., Conclusion: The incidence of youth-onset type 1 and 2 diabetes was significantly different between registries. Additional data are needed to elucidate whether the differences observed represent diagnostic delay, differences in genetic susceptibility, or differences in distribution of risk factors., (© 2020 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)

Background: Over the last decades, diabetes in youth has increased in both India and the United States, along with the burden of long-term complications and healthcare costs. However, there are limited standardized population-based data in contemporary youth cohorts for comparison of clinical and demographic characteristics of diabetes for both type 1 (T1D) and type 2 (T2D)., Methods: In partnership, we harmonized demographic and clinical data from the SEARCH for Diabetes in Youth (SEARCH) registry in the United States and the Registry of People with Diabetes with Youth Age at Onset (YDR) in India to the structure and terminology of the Observational Medical Outcomes Partnership Common Data Model. Data were from youth with T1D and T2D, aged <20 years and newly diagnosed between 2006 and 2010. We compared key characteristics across registries using χ 2 tests and t-tests., Results: In total, there were 9650 youth with T1D and 2406 youth with T2D from 2006 to 2012. SEARCH youth were diagnosed at younger ages than YDR youth for T1D and T2D (10.0 vs 10.5 years, P < .001 and 14.7 vs 16.1 years, P < .001, respectively). For T2D, SEARCH had a higher proportion of females and significantly lower proportion of youth of high socioeconomic status compared to YDR. For T1D and T2D, SEARCH youth had higher BMI, lower blood pressure, and lower A1c compared to YDR youth., Conclusions: These data offer insights into the demographic and clinical characteristics of diabetes in youth across the two countries. Further research is needed to better understand why these differences exist., (© 2020 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)

Background: There is significant global variation in the prevalence of diabetic ketoacidosis (DKA) at diagnosis among youth with type 1 diabetes (T1D). However, data for youth with type 2 diabetes (T2D) are limited, even in developed countries. We compared the prevalence of DKA at diagnosis among individuals with T1D and T2D from the SEARCH for Diabetes in Youth (SEARCH) and the Registry of Youth Onset Diabetes in India (YDR) registries., Methods: We harmonized the SEARCH and YDR registries to the structure and terminology in the Observational Medical Outcome Partnership Common Data Model. Data used were from youth with T1D and T2D diagnosed before 20 years and newly diagnosed between 2006 and 2012 in YDR and 2009 and 2012 in SEARCH., Results: There were 5366 US youth (4078 with T1D, 1288 with T2D) and 2335 Indian youth (2108 with T1D, 227 with T2D). More than one third of T1D youth enrolled in SEARCH had DKA at diagnosis which was significantly higher than in YDR (35.3% vs 28.7%, P < .0001). The burden of DKA in youth with T1D was significantly higher among younger age groups; this relationship was similar across registries (P = .4). The prevalence of DKA among T2D in SEARCH and YDR were 5.5% and 6.6% respectively (P = .4)., Conclusions: There is significant burden of DKA at diagnosis with T1D among youth from United States and India, especially among the younger age groups. The reasons for this high prevalence are largely unknown but are critical to developing interventions to prevent DKA at diagnosis., (© 2020 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)

Objective: To compare treatment regimens and glycosylated hemoglobin (A1c) levels in Type 1 (T1D) and Type 2 diabetes (T2D) using diabetes registries from two countries-U.S. SEARCH for Diabetes in Youth (SEARCH) and Indian Registry of youth onset diabetes in India (YDR)., Methods: The SEARCH and YDR data were harmonized to the structure and terminology in the Observational Medical Outcomes Partnership Common Data Model. Data used were from T1D and T2D youth diagnosed <20 years between 2006-2012 for YDR, and 2006, 2008, and 2012 for SEARCH. We compared treatment regimens and A1c levels across the two registries., Results: There were 4003 T1D (SEARCH = 1899; YDR = 2104) and 611 T2D (SEARCH = 384; YDR = 227) youth. The mean A1c was higher in YDR compared to SEARCH (T1D:11.0% ± 2.9% vs 7.8% ± 1.7%, P < .001; T2D:9.9% ± 2.8% vs 7.2% ± 2.1%, P < .001). Among T1D youth in SEARCH, 65.1% were on a basal/bolus regimen, whereas in YDR, 52.8% were on once/twice daily insulin regimen. Pumps were used by 16.2% of SEARCH and 1.5% of YDR youth with T1D. Among T2D youth, in SEARCH and YDR, a majority were on metformin only (43.0% vs 30.0%), followed by insulin + any oral hypoglycemic agents (26.3% vs 13.7%) and insulin only (12.8% vs 18.9%), respectively., Conclusion: We found significant differences between SEARCH and YDR in treatment patterns in T1D and T2D. A1c levels were higher in YDR than SEARCH youth, for both T1D and T2D, irrespective of the regimens used. Efforts to achieve better glycemic control for youth are urgently needed to reduce the risk of long-term complications., (© 2020 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)

Anomaly detection and explanation in big volumes of real-world medical data, such as those pertaining to COVID-19, pose some challenges. First, we are dealing with time-series data. Typical time-series data describe behavior of a single object over time. In medical data, we are dealing with time-series data belonging to multiple entities. Thus, there may be multiple subsets of records such that records in each subset, which belong to a single entity are temporally dependent, but the records in different subsets are unrelated. Moreover, the records in a subset contain different types of attributes, some of which must be grouped in a particular manner to make the analysis meaningful. Anomaly detection techniques need to be customized for time-series data belonging to multiple entities. Second, anomaly detection techniques fail to explain the cause of outliers to the experts. This is critical for new diseases and pandemics where current knowledge is insufficient. We propose to address these issues by extending our existing work called IDEAL, which is an LSTM-autoencoder based approach for data quality testing of sequential records, and provides explanations of constraint violations in a manner that is understandable to end-users. The extension (1) uses a novel two-level reshaping technique that splits COVID-19 data sets into multiple temporally-dependent subsequences and (2) adds a data visualization plot to further explain the anomalies and evaluate the level of abnormality of subsequences detected by IDEAL. We performed two systematic evaluation studies for our anomalous subsequence detection. One study uses aggregate data, including the number of cases, deaths, recovered, and percentage of hospitalization rate, collected from a COVID tracking project, New York Times, and Johns Hopkins for the same time period. The other study uses COVID-19 patient medical records obtained from Anschutz Medical Center health data warehouse. The results are promising and indicate that our techniques can be used to detect anomalies in large volumes of real-world unlabeled data whose accuracy or validity is unknown., Competing Interests: Conflict of interestOn behalf of all authors, the corresponding author states that there is no conflict of interest., (© The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd 2021.)

Background: Clinical decision support (CDS) design best practices are intended to provide a narrative representation of factors that influence the success of CDS tools. However, they provide incomplete direction on evidence-based implementation principles., Objective: This study aims to describe an integrated approach toward applying an existing implementation science (IS) framework with CDS design best practices to improve the effectiveness, sustainability, and reproducibility of CDS implementations., Methods: We selected the Practical Robust Implementation and Sustainability Model (PRISM) IS framework. We identified areas where PRISM and CDS design best practices complemented each other and defined methods to address each. Lessons learned from applying these methods were then used to further refine the integrated approach., Results: Our integrated approach to applying PRISM with CDS design best practices consists of 5 key phases that iteratively interact and inform each other: multilevel stakeholder engagement, designing the CDS, design and usability testing, thoughtful deployment, and performance evaluation and maintenance. The approach is led by a dedicated implementation team that includes clinical informatics and analyst builder expertise., Conclusions: Integrating PRISM with CDS design best practices extends user-centered design and accounts for the multilevel, interacting, and dynamic factors that influence CDS implementation in health care. Integrating PRISM with CDS design best practices synthesizes the many known contextual factors that can influence the success of CDS tools, thereby enhancing the reproducibility and sustainability of CDS implementations. Others can adapt this approach to their situation to maximize and sustain CDS implementation success., (©Katy E Trinkley, Michael G Kahn, Tellen D Bennett, Russell E Glasgow, Heather Haugen, David P Kao, Miranda E Kroehl, Chen-Tan Lin, Daniel C Malone, Daniel D Matlock. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 29.10.2020.)

Objective: Diabetes surveillance often requires manual medical chart reviews to confirm status and type. This project aimed to create an electronic health record (EHR)-based procedure for improving surveillance efficiency through automation of case identification., Research Design and Methods: Youth (<20 years old) with potential evidence of diabetes ( N = 8,682) were identified from EHRs at three children's hospitals participating in the SEARCH for Diabetes in Youth Study. True diabetes status/type was determined by manual chart reviews. Multinomial regression was compared with an ICD-10 rule-based algorithm in the ability to correctly identify diabetes status and type. Subsequently, the investigators evaluated a scenario of combining the rule-based algorithm with targeted chart reviews where the algorithm performed poorly., Results: The sample included 5,308 true cases (89.2% type 1 diabetes). The rule-based algorithm outperformed regression for overall accuracy (0.955 vs. 0.936). Type 1 diabetes was classified well by both methods: sensitivity ( Se ) (>0.95), specificity ( Sp ) (>0.96), and positive predictive value (PPV) (>0.97). In contrast, the PPVs for type 2 diabetes were 0.642 and 0.778 for the rule-based algorithm and the multinomial regression, respectively. Combination of the rule-based method with chart reviews ( n = 695, 7.9%) of persons predicted to have non-type 1 diabetes resulted in perfect PPV for the cases reviewed while increasing overall accuracy (0.983). The Se , Sp , and PPV for type 2 diabetes using the combined method were ≥0.91., Conclusions: An ICD-10 algorithm combined with targeted chart reviews accurately identified diabetes status/type and could be an attractive option for diabetes surveillance in youth., (© 2020 by the American Diabetes Association.)

Background: The objective was to describe the design of a population-level electronic health record (EHR) and insurance claims-based surveillance system of adolescents and adults with congenital heart defects (CHDs) in Colorado and to evaluate the bias introduced by duplicate cases across data sources., Methods: The Colorado CHD Surveillance System ascertained individuals aged 11-64 years with a CHD based on International Classification of Diseases, Ninth Revision, Clinical Modification diagnostic coding between 2011 and 2013 from a diverse network of health care systems and an All Payer Claims Database (APCD). A probability-based identity reconciliation algorithm identified duplicate cases. Logistic regression was conducted to investigate bias introduced by duplicate cases on the relationship between CHD severity (severe compared to moderate/mild) and adverse outcomes including all-cause mortality, inpatient hospitalization, and major adverse cardiac events (myocardial infarction, congestive heart failure, or cerebrovascular event). Sensitivity analyses were conducted to investigate bias introduced by the sole use or exclusion of APCD data., Results: A total of 12,293 unique cases were identified, of which 3,476 had a within or between data source duplicate. Duplicate cases were more likely to be in the youngest age group and have private health insurance, a severe heart defect, a CHD comorbidity, and higher health care utilization. We found that failure to resolve duplicate cases between data sources would inflate the relationship between CHD severity and both morbidity and mortality outcomes by ~15%. Sensitivity analyses indicate that scenarios in which APCD was excluded from case finding or relied upon as the sole source of case finding would also result in an overestimation of the relationship between a CHD severity and major adverse outcomes., Discussion: Aggregated EHR- and claims-based surveillance systems of adolescents and adults with CHD that fail to account for duplicate records will introduce considerable bias into research findings., Conclusion: Population-level surveillance systems for rare chronic conditions, such as congenital heart disease, based on aggregation of EHR and claims data require sophisticated identity reconciliation methods to prevent bias introduced by duplicate cases., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Objective: The disjointed healthcare system and the nonexistence of a universal patient identifier across systems necessitates accurate record linkage (RL). We aim to describe the implementation and evaluation of a hybrid record linkage method in a statewide surveillance system for congenital heart disease., Materials and Methods: Clear-text personally identifiable information on individuals in the Colorado Congenital Heart Disease surveillance system was obtained from 5 electronic health record and medical claims data sources. Two deterministic methods and 1 probabilistic RL method using first name, last name, social security number, date of birth, and house number were initially implemented independently and then sequentially in a hybrid approach to assess RL performance., Results: 16 480 nonunique individuals with congenital heart disease were ascertained. Deterministic linkage methods, when performed independently, yielded 4505 linked pairs (consisting of 2 records linked together within or across data sources). Probabilistic RL, using 3 initial characters of last name and gender for blocking, yielded 6294 linked pairs when executed independently. Using a hybrid linkage routine resulted in 6451 linkages and an additional 18%-24% correct linked pairs as compared to the independent methods. A hybrid linkage routine resulted in higher recall and F-measure scores compared to probabilistic and deterministic methods performed independently., Discussion: The hybrid approach resulted in increased linkage accuracy and identified pairs of linked record that would have otherwise been missed when using any independent linkage technique., Conclusion: When performing RL within and across disparate data sources, the hybrid RL routine outperformed independent deterministic and probabilistic methods., (© The Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com.)

Background: Congenital heart defects (CHDs), the most common type of birth defect in the United States, are increasing in prevalence in the general population. Though CHD prevalence at birth has been well described in the United States at about 1%, little is known about long-term survival and prevalence of CHDs beyond childhood. This study aimed to estimate the prevalence of CHDs among adolescents and adults in Colorado., Methods: The prevalence of CHDs among adolescents and adults residing in Colorado during 2011 to 2013 was estimated using log-linear capture-recapture methods to account for incomplete case ascertainment. Five case-finding data sources were used for this analysis including electronic health record data from 4 major health systems and a state-legislated all payer claims database., Results: Twelve thousand two hundred ninety-three unique individuals with CHDs (2481 adolescents and 9812 adults) were identified in one or more primary data sources. We estimated the crude prevalence of CHDs in adolescents and adults in Colorado to be 3.22 per 1000 individuals (95% CI 3.19-3.53). After accounting for incomplete case ascertainment, the final capture-recapture model yielded an estimated total adolescent and adult CHD population of 23,194 (95% CI 22,419-23,565) and an adjusted prevalence of 6.07 per 1000 individuals (95% CI 5.86-6.16), indicating 47% of the cases in the catchment area were not identified in the case-identifying data sources., Conclusion: This statewide study yielded new information on the prevalence of CHDs in adolescents and adults. These high prevalence rates underscore the need for additional specialized care facilities for this population with CHDs., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Electronic health records (EHRs) provide an alternative to traditional public health surveillance surveys and administrative data for measuring the prevalence and impact of chronic health conditions in populations. As the infrastructure for secondary use of EHR data improves, many stakeholders are poised to benefit from data partnerships for regional access to information. Electronic health records can be transformed into a common data model that facilitates data sharing across multiple organizations and allows data to be used for surveillance. The Colorado Health Observation Regional Data Service, a regional distributed data network, has assembled diverse data partnerships, flexible infrastructure, and transparent governance practices to better understand the health of communities through EHR-based, public health surveillance. This article describes attributes of regional distributed data networks using EHR data and the history and design of Colorado Health Observation Regional Data Service as an emerging public health surveillance tool for chronic health conditions. Colorado Health Observation Regional Data Service and our experience may serve as a model for other regions interested in similar surveillance efforts. While benefits from EHR-based surveillance are described, a number of technology, partnership, and value proposition challenges remain.

Introduction: In aggregate, existing data quality (DQ) checks are currently represented in heterogeneous formats, making it difficult to compare, categorize, and index checks. This study contributes a data element-function conceptual model to facilitate the categorization and indexing of DQ checks and explores the feasibility of leveraging natural language processing (NLP) for scalable acquisition of knowledge of common data elements and functions from DQ checks narratives., Methods: The model defines a "data element", the primary focus of the check, and a "function", the qualitative or quantitative measure over a data element. We applied NLP techniques to extract both from 172 checks for Observational Health Data Sciences and Informatics (OHDSI) and 3,434 checks for Kaiser Permanente's Center for Effectiveness and Safety Research (CESR)., Results: The model was able to classify all checks. A total of 751 unique data elements and 24 unique functions were extracted. The top five frequent data element-function pairings for OHDSI were Person-Count (55 checks), Insurance-Distribution (17), Medication-Count (16), Condition-Count (14), and Observations-Count (13); for CESR, they were Medication-Variable Type (175), Medication-Missing (172), Medication-Existence (152), Medication-Count (127), and Socioeconomic Factors-Variable Type (114)., Conclusions: This study shows the efficacy of the data element-function conceptual model for classifying DQ checks, demonstrates early promise of NLP-assisted knowledge acquisition, and reveals the great heterogeneity in the focus in DQ checks, confirming variation in intrinsic checks and use-case specific "fitness-for-use" checks., Competing Interests: The authors have no competing interests to declare.

The vision of precision medicine relies on the integration of large-scale clinical, molecular and environmental datasets. Data integration may be thought of along two axes: data fusion across institutions, and data fusion across modalities. Cross-institutional data sharing that maintains semantic integrity hinges on the adoption of data standards and a push toward ontology-driven integration. The goal should be the creation of query-able data repositories spanning primary and tertiary care providers, disease registries, research organizations etc. to produce rich longitudinal datasets. Cross-modality sharing involves the integration of multiple data streams, from structured EHR data (diagnosis codes, laboratory tests) to genomics, imaging, monitors and patient-generated data including wearable devices. This integration presents unique technical, semantic, and ethical challenges; however recent work suggests that multi-modal clinical data can significantly improve the performance of phenotyping and prediction algorithms, powering knowledge discovery at the patient- and population-level.

Background: The electronic health record (EHR) was designed as a clinical and administrative tool to improve clinical patient care. Electronic healthcare systems have been successfully adopted across the world through use of government mandates and incentives., Methods: Using electronic health record, health information system, electronic medical record, health information systems, research, outcomes, pediatric, surgery, and urology as initial search terms, the literature focusing on clinical documentation data capture and the EHR as a potential resource for research related to clinical outcomes, quality improvement, and comparative effectiveness was reviewed. Relevant articles were supplemented by secondary review of article references as well as seminal articles in the field as identified by the senior author., Findings: US federal funding agencies, including the Agency for Healthcare Research and Quality, the Patient-Centered Outcomes Research Institute, the National Institutes of Health, and the Food and Drug Administration have recognized the EHR's role supporting research. The main approached to using EHR data include enhanced lists, direct data extraction, structured data entry, and unstructured data entry. The EHR's potential to facilitate research, overcoming cost and time burdens associated with traditional data collection, has not resulted in widespread use of EHR-based research tools., Conclusion: There are strengths and weaknesses for all existing methodologies of using EHR data to support research. Collaboration is needed to identify the method that best suits the institution for incorporation of research-oriented data collection into routine pediatric urologic clinical practice., (Copyright © 2018 Journal of Pediatric Urology Company. Published by Elsevier Ltd. All rights reserved.)

Objective: PEDSnet is a clinical data research network (CDRN) that aggregates electronic health record data from multiple children's hospitals to enable large-scale research. Assessing data quality to ensure suitability for conducting research is a key requirement in PEDSnet. This study presents a range of data quality issues identified over a period of 18 months and interprets them to evaluate the research capacity of PEDSnet., Materials and Methods: Results were generated by a semiautomated data quality assessment workflow. Two investigators reviewed programmatic data quality issues and conducted discussions with the data partners' extract-transform-load analysts to determine the cause for each issue., Results: The results include a longitudinal summary of 2182 data quality issues identified across 9 data submission cycles. The metadata from the most recent cycle includes annotations for 850 issues: most frequent types, including missing data (>300) and outliers (>100); most complex domains, including medications (>160) and lab measurements (>140); and primary causes, including source data characteristics (83%) and extract-transform-load errors (9%)., Discussion: The longitudinal findings demonstrate the network's evolution from identifying difficulties with aligning the data to a common data model to learning norms in clinical pediatrics and determining research capability., Conclusion: While data quality is recognized as a critical aspect in establishing and utilizing a CDRN, the findings from data quality assessments are largely unpublished. This paper presents a real-world account of studying and interpreting data quality findings in a pediatric CDRN, and the lessons learned could be used by other CDRNs., (© The Author, 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com)

Background: Electronic health records (EHRs) contain detailed clinical data stored in proprietary formats with non-standard codes and structures. Participating in multi-site clinical research networks requires EHR data to be restructured and transformed into a common format and standard terminologies, and optimally linked to other data sources. The expertise and scalable solutions needed to transform data to conform to network requirements are beyond the scope of many health care organizations and there is a need for practical tools that lower the barriers of data contribution to clinical research networks., Methods: We designed and implemented a health data transformation and loading approach, which we refer to as Dynamic ETL (Extraction, Transformation and Loading) (D-ETL), that automates part of the process through use of scalable, reusable and customizable code, while retaining manual aspects of the process that requires knowledge of complex coding syntax. This approach provides the flexibility required for the ETL of heterogeneous data, variations in semantic expertise, and transparency of transformation logic that are essential to implement ETL conventions across clinical research sharing networks. Processing workflows are directed by the ETL specifications guideline, developed by ETL designers with extensive knowledge of the structure and semantics of health data (i.e., "health data domain experts") and target common data model., Results: D-ETL was implemented to perform ETL operations that load data from various sources with different database schema structures into the Observational Medical Outcome Partnership (OMOP) common data model. The results showed that ETL rule composition methods and the D-ETL engine offer a scalable solution for health data transformation via automatic query generation to harmonize source datasets., Conclusions: D-ETL supports a flexible and transparent process to transform and load health data into a target data model. This approach offers a solution that lowers technical barriers that prevent data partners from participating in research data networks, and therefore, promotes the advancement of comparative effectiveness research using secondary electronic health data.

Pragmatic clinical trials (PCTs) are research investigations embedded in health care settings designed to increase the efficiency of research and its relevance to clinical practice. The Health Care Systems Research Collaboratory, initiated by the National Institutes of Health Common Fund in 2010, is a pioneering cooperative aimed at identifying and overcoming operational challenges to pragmatic research. Drawing from our experience, we present 4 broad categories of informatics-related challenges: (1) using clinical data for research, (2) integrating data from heterogeneous systems, (3) using electronic health records to support intervention delivery or health system change, and (4) assessing and improving data capture to define study populations and outcomes. These challenges impact the validity, reliability, and integrity of PCTs. Achieving the full potential of PCTs and a learning health system will require meaningful partnerships between health system leadership and operations, and federally driven standards and policies to ensure that future electronic health record systems have the flexibility to support research., (© The Author 2017. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: journals.permissions@oup.com.)

Background: Contributing health data to national, regional, and local networks or registries requires data stored in local systems with local structures and codes to be extracted, transformed, and loaded into a standard format called a Common Data Model (CDM). These processes called Extract, Transform, Load (ETL) require data partners or contributors to invest in costly technical resources with specialized skills in data models, terminologies, and programming. Given the wide range of tasks, skills, and technologies required to transform data into a CDM, a classification of ETL challenges can help identify needed resources, which in turn may encourage data partners with less-technical capabilities to participate in data-sharing networks., Methods: We conducted key-informant interviews with data partner representatives to survey the ETL challenges faced in clinical data research networks (CDRNs) and registries. A list of ETL challenges, organized into six themes was vetted during a one-day workshop with a wide range of network stakeholders including data partners, researchers, and policy experts., Results: We identified 24 technical ETL challenges related to the data sharing process. All of these ETL challenges were rated as "important" or "very important" by workshop participants using a five point Likert scale. Based on these findings, a framework for categorizing ETL challenges according to ETL phases, themes, and levels of data network participation was developed., Conclusions: Overcoming ETL technical challenges require significant investments in a broad array of information technologies and human resources. Identifying these technical obstacles can inform optimal resource allocation to minimize the barriers and cost of entry for new data partners into extant networks, which in turn can expand data networks' inclusiveness and diversity. This paper offers pertinent information and guiding framework that are relevant for data partners in ascertaining challenges associated with contributing data in data networks.

Objective: To compare rule-based data quality (DQ) assessment approaches across multiple national clinical data sharing organizations., Methods: Six organizations with established data quality assessment (DQA) programs provided documentation or source code describing current DQ checks. DQ checks were mapped to the categories within the data verification context of the harmonized DQA terminology. To ensure all DQ checks were consistently mapped, conventions were developed and four iterations of mapping performed. Difficult-to-map DQ checks were discussed with research team members until consensus was achieved., Results: Participating organizations provided 11,026 DQ checks, of which 99.97 percent were successfully mapped to a DQA category. Of the mapped DQ checks (N=11,023), 214 (1.94 percent) mapped to multiple DQA categories. The majority of DQ checks mapped to Atemporal Plausibility (49.60 percent), Value Conformance (17.84 percent), and Atemporal Completeness (12.98 percent) categories., Discussion: Using the common DQA terminology, near-complete (99.97 percent) coverage across a wide range of DQA programs and specifications was reached. Comparing the distributions of mapped DQ checks revealed important differences between participating organizations. This variation may be related to the organization's stakeholder requirements, primary analytical focus, or maturity of their DQA program. Not within scope, mapping checks within the data validation context of the terminology may provide additional insights into DQA practice differences., Conclusion: A common DQA terminology provides a means to help organizations and researchers understand the coverage of their current DQA efforts as well as highlight potential areas for additional DQA development. Sharing DQ checks between organizations could help expand the scope of DQA across clinical data networks.

Objectives: To reflect on the notable events and significant developments in Clinical Research Informatics (CRI) in the year of 2015 and discuss near-term trends impacting CRI., Methods: We selected key publications that highlight not only important recent advances in CRI but also notable events likely to have significant impact on CRI activities over the next few years or longer, and consulted the discussions in relevant scientific communities and an online living textbook for modern clinical trials. We also related the new concepts with old problems to improve the continuity of CRI research., Results: The highlights in CRI in 2015 include the growing adoption of electronic health records (EHR), the rapid development of regional, national, and global clinical data research networks for using EHR data to integrate scalable clinical research with clinical care and generate robust medical evidence. Data quality, integration, and fusion, data access by researchers, study transparency, results reproducibility, and infrastructure sustainability are persistent challenges., Conclusion: The advances in Big Data Analytics and Internet technologies together with the engagement of citizens in sciences are shaping the global clinical research enterprise, which is getting more open and increasingly stakeholder-centered, where stakeholders include patients, clinicians, researchers, and sponsors.

Objective: Harmonized data quality (DQ) assessment terms, methods, and reporting practices can establish a common understanding of the strengths and limitations of electronic health record (EHR) data for operational analytics, quality improvement, and research. Existing published DQ terms were harmonized to a comprehensive unified terminology with definitions and examples and organized into a conceptual framework to support a common approach to defining whether EHR data is 'fit' for specific uses., Materials and Methods: DQ publications, informatics and analytics experts, managers of established DQ programs, and operational manuals from several mature EHR-based research networks were reviewed to identify potential DQ terms and categories. Two face-to-face stakeholder meetings were used to vet an initial set of DQ terms and definitions that were grouped into an overall conceptual framework. Feedback received from data producers and users was used to construct a draft set of harmonized DQ terms and categories. Multiple rounds of iterative refinement resulted in a set of terms and organizing framework consisting of DQ categories, subcategories, terms, definitions, and examples. The harmonized terminology and logical framework's inclusiveness was evaluated against ten published DQ terminologies., Results: Existing DQ terms were harmonized and organized into a framework by defining three DQ categories: (1) Conformance (2) Completeness and (3) Plausibility and two DQ assessment contexts: (1) Verification and (2) Validation. Conformance and Plausibility categories were further divided into subcategories. Each category and subcategory was defined with respect to whether the data may be verified with organizational data, or validated against an accepted gold standard, depending on proposed context and uses. The coverage of the harmonized DQ terminology was validated by successfully aligning to multiple published DQ terminologies., Discussion: Existing DQ concepts, community input, and expert review informed the development of a distinct set of terms, organized into categories and subcategories. The resulting DQ terms successfully encompassed a wide range of disparate DQ terminologies. Operational definitions were developed to provide guidance for implementing DQ assessment procedures. The resulting structure is an inclusive DQ framework for standardizing DQ assessment and reporting. While our analysis focused on the DQ issues often found in EHR data, the new terminology may be applicable to a wide range of electronic health data such as administrative, research, and patient-reported data., Conclusion: A consistent, common DQ terminology, organized into a logical framework, is an initial step in enabling data owners and users, patients, and policy makers to evaluate and communicate data quality findings in a well-defined manner with a shared vocabulary. Future work will leverage the framework and terminology to develop reusable data quality assessment and reporting methods.

Introduction: An often key component to coordinating surveillance activities across distributed networks is the design and implementation of a common data model (CDM). The purpose of this study was to evaluate two drug safety surveillance CDMs from an ecosystem perspective to better understand how differences in CDMs and analytic tools affect usability and interpretation of results., Methods: Humana claims data from 2007 to 2012 were mapped to Observational Medical Outcomes Partnership (OMOP) and Mini-Sentinel CDMs. Data were described and compared at the patient level by source code and mapped concepts. Study cohort construction and effect estimates were also compared using two different analytical methods--one based on a new user design implementing a high-dimensional propensity score (HDPS) algorithm and the other based on univariate self-controlled case series (SCCS) design--across six established positive drug-outcome pairs to learn how differences in CDMs and analytics influence steps in the database analytic process and results., Results: Claims data for approximately 7.7 million Humana health plan members were transformed into the two CDMs. Three health outcome cohorts and two drug cohorts showed differences in cohort size and constituency between Mini-Sentinel and OMOP CDMs, which was a result of multiple factors. Overall, the implementation of the HDPS procedure on Mini-Sentinel CDM detected more known positive associations than that on OMOP CDM. The SCCS method results were comparable on both CDMs. Differences in the implementation of the HDPS procedure between the two CDMs were identified; analytic model and risk period specification had a significant impact on the performance of the HDPS procedure on OMOP CDM., Conclusions: Differences were observed between OMOP and Mini-Sentinel CDMs. The analysis of both CDMs at the data model level indicated that such conceptual differences had only a slight but not significant impact on identifying known safety associations. Our results show that differences at the ecosystem level of analyses across the CDMs can lead to strikingly different risk estimations, but this can be primarily attributed to the choices of analytic approach and their implementation in the community-developed analytic tools. The opportunities of using CDMs are clear, but our study shows the need for judicious comparison of analyses across the CDMs. Our work emphasizes the need for ongoing efforts to ensure sustainable transparent platforms to maintain and develop CDMs and associated tools for effective safety surveillance.