Search

PurposeElectronic health records (EHRs) contain a vast amount of clinical data. Improved automated classification approaches have the potential to accurately and efficiently identify patient cohorts for research. We evaluated if a rule-based natural language processing (NLP) algorithm using clinical notes performed better for classifying proliferative diabetic retinopathy (PDR) and nonproliferative diabetic retinopathy (NPDR) severity compared with International Classification of Diseases, ninth edition (ICD-9) or 10th edition (ICD-10) codes.DesignCross-sectional study.SubjectsDeidentified EHR data from an academic medical center identified 2366 patients aged ≥18 years, with diabetes mellitus, diabetic retinopathy (DR), and available clinical notes.MethodsFrom these 2366 patients, 306 random patients (100 training set, 206 test set) underwent chart review by ophthalmologists to establish the gold standard. International Classification of Diseases codes were extracted from the EHR. The notes algorithm identified positive mention of PDR and NPDR severity from clinical notes. Proliferative diabetic retinopathy and NPDR severity classification by ICD codes and the notes algorithm were compared with the gold standard. The entire DR cohort (N = 2366) was then classified as having presence (or absence) of PDR using ICD codes and the notes algorithm.Main outcome measuresSensitivity, specificity, positive predictive value (PPV), negative predictive value, and F1 score for the notes algorithm compared with ICD codes using a gold standard of chart review.ResultsFor PDR classification of the test set patients, the notes algorithm performed better than ICD codes for all metrics. Specifically, the notes algorithm had significantly higher sensitivity (90.5% [95% confidence interval 85.7, 94.9] vs. 68.4% [60.4, 75.3]), but similar PPV (98.0% [95.4-100] vs. 94.7% [90.3, 98.3]) respectively. The F1 score was 0.941 [0.910, 0.966] for the notes algorithm compared with 0.794 [0.734, 0.842] for ICD codes. For PDR classification, ICD-10 codes performed better than ICD-9 codes (F1 score 0.836 [0.771, 0.878] vs. 0.596 [0.222, 0.692]). For NPDR severity classification, the notes algorithm performed similarly to ICD codes, but performance was limited by small sample size.ConclusionsThe notes algorithm outperformed ICD codes for PDR classification. The findings demonstrate the significant potential of applying a rule-based NLP algorithm to clinical notes to increase the efficiency and accuracy of cohort selection for research.Financial disclosuresProprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

PurposeUse of the electronic health record (EHR) has motivated the need for data standardization. A gap in knowledge exists regarding variations in existing terminologies for defining diabetic retinopathy (DR) cohorts. This study aimed to review the literature and analyze variations regarding codified definitions of DR.DesignLiterature review and quantitative analysis.SubjectsPublished manuscripts.MethodsFour graders reviewed PubMed and Google Scholar for peer-reviewed studies. Studies were included if they used codified definitions of DR (e.g., billing codes). Data elements such as author names, publication year, purpose, data set type, and DR definitions were manually extracted. Each study was reviewed by ≥ 2 authors to validate inclusion eligibility. Quantitative analyses of the codified definitions were then performed to characterize the variation between DR cohort definitions.Main outcome measuresNumber of studies included and numeric counts of billing codes used to define codified cohorts.ResultsIn total, 43 studies met the inclusion criteria. Half of the included studies used datasets based on structured EHR data (i.e., data registries, institutional EHR review), and half used claims data. All but 1 of the studies used billing codes such as the International Classification of Diseases 9th or 10th edition (ICD-9 or ICD-10), either alone or in addition to another terminology for defining disease. Of the 27 included studies that used ICD-9 and the 20 studies that used ICD-10 codes, the most common codes used pertained to the full spectrum of DR severity. Diabetic retinopathy complications (e.g., vitreous hemorrhage) were also used to define some DR cohorts.ConclusionsSubstantial variations exist among codified definitions for DR cohorts within retrospective studies. Variable definitions may limit generalizability and reproducibility of retrospective studies. More work is needed to standardize disease cohorts.Financial disclosuresProprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

ObjectiveTo determine if baseline diabetic retinopathy (DR) severity mediates the relationship between health insurance status and DR progression.DesignRetrospective cohort study.SubjectsSeven hundred sixteen patients aged ≥ 18 years with a diagnosis of type 1 or 2 diabetes mellitus, and a diagnosis of nonproliferative DR (NPDR) were identified from the electronic health record of a tertiary academic center between June 2012 and February 2022.MethodsNPDR severity at baseline was the proposed mediator in the relationship between insurance status and proliferative DR (PDR) progression. Logistic regression was used to determine the association between insurance status and NPDR severity at baseline, and Cox proportional hazards regression was used to assess the association between insurance status and time to PDR progression. To analyze the mediation effect of NPDR severity at baseline, a counterfactual approach, which decomposes a total effect into a natural direct effect and a natural indirect effect was applied.Main outcome measuresTime to progression from first NPDR diagnosis to first PDR diagnosis.ResultsOf the 716 patients, 581 (81%) had Medicare or private insurance, 107 (15%) had Medicaid, and 28 (4.0%) were uninsured at their baseline eye visit. Uninsured or Medicaid patients had a higher proportion of moderate or severe NPDR at their baseline eye visit and a higher proportion of progression to PDR. After adjusting for confounders and NPDR severity at baseline, patients who were uninsured had significantly greater risk of progression to PDR compared with that of patients with Medicare/private insurance (hazard ratio [HR]: 2.63; 95% confidence interval [CI]: 1.10-6.25). Patients with Medicaid also had an increased risk of progression to PDR compared with that of patients with Medicare/private insurance, although not statistically significant (HR: 1.53; 95% CI: 0.81-2.89). NPDR severity at baseline mediated 41% of the effect of insurance status (uninsured vs. Medicare/private insurance) on PDR progression.ConclusionsPatients who were uninsured were more likely to have an advanced stage of NPDR at their baseline eye visit and were at significantly greater risk of progression to PDR compared with patients who had Medicare or were privately insured. Mediation analysis revealed that differences in baseline NPDR severity by insurance explained a significant proportion of the relationship between insurance status and DR progression.Financial disclosuresProprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.