Search

Abstract Importance Large language models (LLMs) like OpenAI’s ChatGPT are powerful generative systems that rapidly synthesize natural language responses. Research on LLMs has revealed their potential and pitfalls, especially in clinical settings. However, the evolving landscape of LLM research in medicine has left several gaps regarding their evaluation, application, and evidence base. Objective This scoping review aims to (1) summarize current research evidence on the accuracy and efficacy of LLMs in medical applications, (2) discuss the ethical, legal, logistical, and socioeconomic implications of LLM use in clinical settings, (3) explore barriers and facilitators to LLM implementation in healthcare, (4) propose a standardized evaluation framework for assessing LLMs’ clinical utility, and (5) identify evidence gaps and propose future research directions for LLMs in clinical applications. Evidence review We screened 4,036 records from MEDLINE, EMBASE, CINAHL, medRxiv, bioRxiv, and arXiv from January 2023 (inception of the search) to June 26, 2023 for English-language papers and analyzed findings from 55 worldwide studies. Quality of evidence was reported based on the Oxford Centre for Evidence-based Medicine recommendations. Findings Our results demonstrate that LLMs show promise in compiling patient notes, assisting patients in navigating the healthcare system, and to some extent, supporting clinical decision-making when combined with human oversight. However, their utilization is limited by biases in training data that may harm patients, the generation of inaccurate but convincing information, and ethical, legal, socioeconomic, and privacy concerns. We also identified a lack of standardized methods for evaluating LLMs’ effectiveness and feasibility. Conclusions and relevance This review thus highlights potential future directions and questions to address these limitations and to further explore LLMs’ potential in enhancing healthcare delivery.

Abstract Background Non-alcoholic fatty liver disease (NAFLD) describes a spectrum of chronic fattening of liver that can lead to fibrosis and cirrhosis. Diabetes has been identified as a major comorbidity that contributes to NAFLD progression. Health systems around the world make use of administrative data to conduct population-based prevalence studies. To that end, we sought to assess the accuracy of diabetes International Classification of Diseases (ICD) coding in administrative databases among a cohort of confirmed NAFLD patients in Calgary, Alberta, Canada. Methods The Calgary NAFLD Pathway Database was linked to the following databases: Physician Claims, Discharge Abstract Database, National Ambulatory Care Reporting System, Pharmaceutical Information Network database, Laboratory, and Electronic Medical Records. Hemoglobin A1c and diabetes medication details were used to classify diabetes groups into absent, prediabetes, meeting glycemic targets, and not meeting glycemic targets. The performance of ICD codes among these groups was compared to this standard. Within each group, the total numbers of true positives, false positives, false negatives, and true negatives were calculated. Descriptive statistics and bivariate analysis were conducted on identified covariates, including demographics and types of interacted physicians. Results A total of 12,012 NAFLD patients were registered through the Calgary NAFLD Pathway Database and 100% were successfully linked to the administrative databases. Overall, diabetes coding showed a sensitivity of 0.81 and a positive predictive value of 0.87. False negative rates in the absent and not meeting glycemic control groups were 4.5% and 6.4%, respectively, whereas the meeting glycemic control group had a 42.2% coding error. Visits to primary and outpatient services were associated with most encounters. Conclusion Diabetes ICD coding in administrative databases can accurately detect true diabetic cases. However, patients with diabetes who meets glycemic control targets are less likely to be coded in administrative databases. A detailed understanding of the clinical context will require additional data linkage from primary care settings.

Abstract Background Parents of infants in neonatal intensive care units (NICUs) are often unintentionally marginalized in pursuit of optimal clinical care. Family Integrated Care (FICare) was developed to support families as part of their infants’ care team in level III NICUs. We adapted the model for level II NICUs in Alberta, Canada, and evaluated whether the new Alberta FICare™ model decreased hospital length of stay (LOS) in preterm infants without concomitant increases in readmissions and emergency department visits. Methods In this pragmatic cluster randomized controlled trial conducted between December 15, 2015 and July 28, 2018, 10 level II NICUs were randomized to provide Alberta FICare™ (n = 5) or standard care (n = 5). Alberta FICare™ is a psychoeducational intervention with 3 components: Relational Communication, Parent Education, and Parent Support. We enrolled mothers and their singleton or twin infants born between 32 0/7 and 34 6/7 weeks gestation. The primary outcome was infant hospital LOS. We used a linear regression model to conduct weighted site-level analysis comparing adjusted mean LOS between groups, accounting for site geographic area (urban/regional) and infant risk factors. Secondary outcomes included proportions of infants with readmissions and emergency department visits to 2 months corrected age, type of feeding at discharge, and maternal psychosocial distress and parenting self-efficacy at discharge. Results We enrolled 654 mothers and 765 infants (543 singletons/111 twin cases). Intention to treat analysis included 353 infants/308 mothers in the Alberta FICare™ group and 365 infants/306 mothers in the standard care group. The unadjusted difference between groups in infant hospital LOS (1.96 days) was not statistically significant. Accounting for site geographic area and infant risk factors, infant hospital LOS was 2.55 days shorter (95% CI, − 4.44 to − 0.66) in the Alberta FICare™ group than standard care group, P = .02. Secondary outcomes were not significantly different between groups. Conclusions Alberta FICare™ is effective in reducing preterm infant LOS in level II NICUs, without concomitant increases in readmissions or emergency department visits. A small number of sites in a single jurisdiction and select group infants limit generalizability of findings. Trial registration ClinicalTrials.gov Identifier NCT02879799 , retrospectively registered August 26, 2016.

Abstract Background The first Canadian outbreak of community-associated methicillin-resistant Staphylococcus aureus (CA-MRSA) was identified in 2004 in Calgary, Alberta. Using a novel model of MRSA population-based surveillance, sociodemographic risk associations, yearly geospatial dissemination and prevalence of CA-MRSA infections over an 11 year period was identified in an urban healthcare jurisdiction of Calgary. Methods Positive MRSA case records, patient demographics and laboratory data were obtained from a centralized Laboratory Information System of Calgary Laboratory Services in Calgary, Alberta, Canada between 2004 and 2014. Public census data was obtained from Statistics Canada, which was used to match with laboratory data and mapped using Geographic Information Systems. Results During the study period, 52.5% of positive MRSA infections in Calgary were CA-MRSA cases. The majority were CMRSA10 (USA300) clones (94.1%; n = 4255), while the remaining case (n = 266) were CMRSA7 (USA400) clones. Period prevalence of CMRSA10 increased from 3.6 cases/100000 population in 2004, to 41.3 cases/100000 population in 2014. Geospatial analysis demonstrated wide dissemination of CMRSA10 annually in the city. Those who are English speaking (RR = 0.05, p

Abstract Objective The incidence of the combined myeloproliferative neoplasms (MPNs) was determined for a major Canadian city. Retrospective cases of MPN diagnoses (essential thrombocythemia, polycythemia vera, and primary myelofibrosis) between 2011 to 2015 were retrieved from the Southern Alberta Cancer Cytogenetics Laboratory’s database at Alberta Public Laboratories. Results An incidence rate of 2.05 cases per 100,000 person-years (95% CI 1.73–2.41) was determined, giving an age-standardized Canadian incidence of 2.71 cases per 100,000 person years (95% CI 2.63–2.78). MPN diagnoses occurred at a wide age range of 8–93 (median 66) and an age-dependent increase in incidence. Incidence rates for the MPNs are first reported here for a Canadian population.

Abstract Objective The epidemiology of chronic myeloid leukemia is shifting due to the aging global population and the recent discovery and availability of targeted treatment options. This study provides recent data regarding the incidence of CML in Calgary, a major Canadian city. Data from patients diagnosed with CML by bone marrow sample analysis from 2011 to 2015 were collected from the database of the sole centralized cytogenetics facility in service of Calgary and its surrounding area. Results With an average of 10.2 newly diagnosed cases per year in Calgary from 2011 to 2015, the incidence rate was calculated to be 0.75 cases per 100,000 person-years (95% CI 0.57–0.99). With age standardization, the incidence was 0.87 cases per 100,000 person-years (95% CI 0.82–0.91) for the Canadian population, which was low compared to other developed Western nations. The highest incidence rates were observed in the older patient categories, however there was a broad age distribution for incident cases and the median age at diagnosis was 48. There was a general male bias for CML most pronounced at the younger ages. Our description of CML incidence will help to inform healthcare planners amidst the dramatically altered treatment of this hematological neoplasm.

Abstract Background Chronic kidney disease (CKD) is often asymptomatic in its early stages but is indicated and is diagnosed with an estimated glomerular filtration rate (eGFR)

Abstract Objective Acute lymphocytic leukemia (ALL) is a rare malignant neoplasm that develops from abnormal lymphoid stem cells. ALL incidence is highest among children and declines towards adolescence. There is limited data on the epidemiology of ALL, especially in Canada. This retrospective cohort study used patient data from the Calgary Laboratory Services Cancer Cytogenetics Laboratory to report the incidence rate of ALL in Calgary, Alberta, Canada. New cases of ALL were identified for the 5-year period of January 1, 2011 until December 31, 2015. Reported incidence rates were categorized by sex and age groups, and age-standardized to the Canadian population. Results There were an average of 11.4 new cases of ALL diagnosed per year between 2011 and 2015. The total incidence rate per 100,000 person-years was 0.84. Incidence rates peaked in children aged 0–4 with 7.55 and 3.32 cases per 100,000 person-years for males and females, respectively. The median age of diagnosis was 8 years. Incidence rates were generally lowest for adults aged 20 and over. The ratio of males to females diagnosed with ALL was 1.59. Overall, the recent incidence of ALL in Calgary is comparatively low with a preference for males and children below 5 years of age.

Abstract Background Many clinical practice guidelines encourage diagnosis and empiric treatment of lower urinary tract infection without laboratory investigation; however, urine culture testing remains one of the largest volume tests in the clinical microbiology laboratory. In this study, we sought to determine if there were specific patient groups to which increased testing was directed. To do so, we combined laboratory data on testing rates with Census Canada sociodemographic data. Methods Urine culture testing data was obtained from the Calgary Laboratory Services information system for 2011. We examined all census dissemination areas within the city of Calgary and, for each area, testing rates were determined for age and gender cohorts. We then compared these testing rates to sociodemographic factors obtained from Census Canada and used Poisson regression and generalized estimating equations to test associations between testing rates and sociodemographic variables. Results Per capita urine culture testing is increasing in Calgary. For 2011, 100,901 individuals (9.2% of all people) received urine cultures and were included in this analysis. The majority of cultures were received from the community (67.9%). Substantial differences in rate of testing were observed across the city. Most notably, urine culture testing was drastically lower in areas of high (≥ $100000) household income (RR = 0.07, p

Abstract Background Every year, about 15 million of the world’s infants are born preterm (before 37 weeks gestation). In Alberta, the preterm birth rate was 8.7% in 2015, the second highest among Canadian provinces. Approximately 20% of preterm infants are born before 32 weeks gestation (early preterm), and require care in a Level III neonatal intensive care unit (NICU); 80% are born moderate (32 weeks and zero days [320/7] to 336/7 weeks) and late preterm (340/7 to 366/7 weeks), and require care in a Level II NICU. Preterm birth and experiences in the NICU disrupt early parent-infant relationships and induce parental psychosocial distress. Family Integrated Care (FICare) shows promise as a model of care in Level III NICUs. The purpose of this study is to evaluate length of stay, infant and maternal clinical outcomes, and costs following adaptation and implementation of FICare in Level II NICUs. Methods We will conduct a pragmatic, cluster randomized controlled trial (cRCT) in ten Alberta Level II NICUs allocated to one of two groups: FICare or standard care. The FICare Alberta model involves three theoretically-based, standardized components: information sharing, parenting education, and family support. Our sample size of 181 mother-infant dyads per group is based on the primary outcome of NICU length of stay, 80% participation, and 80% retention at follow-up. Secondary outcomes (e.g., infant clinical outcomes and maternal psychosocial distress) will be assessed shortly after admission to NICU, at discharge and 2 months corrected age. We will conduct economic analysis from two perspectives: the public healthcare payer and society. To understand the utility, acceptability, and impact of FICare, qualitative interviews will be conducted with a subset of mothers at the 2-month follow-up, and with hospital administrators and healthcare providers near the end of the study. Discussion Results of this pragmatic cRCT of FICare in Alberta Level II NICUs will inform policy decisions by providing evidence about the clinical effectiveness and costs of FICare. Trial registration ClinicalTrials.gov, ID: NCT02879799 . Registered on 27 May 2016. Protocol version: 9 June 2016; version 2.

Abstract Background The incidence rate of acute myeloid leukemia (AML) was determined in the Calgary Metropolitan Area, a major Canadian city. Methods Data from all patients diagnosed with AML between January 1, 2011 and December 31, 2015 were retrieved from a single, centralized cancer cytogenetics laboratory for bone marrow samples, the sole diagnostic facility of its kind in Southern Alberta. Results The calculated incidence rate was 2.79 cases per 100,000 person-years with a median age of 60, slightly lower than previously published data. The age-standardized incidence rate for Canada was 3.46 cases per 100,000 person-years. The higher value is reflective of Calgary’s younger population compared to the rest of Canada. Higher male incidence and greatest incidence occurring at approximately the age of 85 is similar to data from other developed countries. The lower incidence rates and median age of diagnosis, in comparison with that of other high-income nations, may be due to differences in the proportion of aging citizens in the population. Conclusion This is the first published incidence rate of acute myeloid leukemia (AML) in Canada across all age groups.

After publication of our article [1], the authors have reported mathematical errors made in the sample size calculation for this cluster randomized controlled trial (cRCT) (Benzies et al. 2017).