Search

Abstract Importance Large language models (LLMs) like OpenAI’s ChatGPT are powerful generative systems that rapidly synthesize natural language responses. Research on LLMs has revealed their potential and pitfalls, especially in clinical settings. However, the evolving landscape of LLM research in medicine has left several gaps regarding their evaluation, application, and evidence base. Objective This scoping review aims to (1) summarize current research evidence on the accuracy and efficacy of LLMs in medical applications, (2) discuss the ethical, legal, logistical, and socioeconomic implications of LLM use in clinical settings, (3) explore barriers and facilitators to LLM implementation in healthcare, (4) propose a standardized evaluation framework for assessing LLMs’ clinical utility, and (5) identify evidence gaps and propose future research directions for LLMs in clinical applications. Evidence review We screened 4,036 records from MEDLINE, EMBASE, CINAHL, medRxiv, bioRxiv, and arXiv from January 2023 (inception of the search) to June 26, 2023 for English-language papers and analyzed findings from 55 worldwide studies. Quality of evidence was reported based on the Oxford Centre for Evidence-based Medicine recommendations. Findings Our results demonstrate that LLMs show promise in compiling patient notes, assisting patients in navigating the healthcare system, and to some extent, supporting clinical decision-making when combined with human oversight. However, their utilization is limited by biases in training data that may harm patients, the generation of inaccurate but convincing information, and ethical, legal, socioeconomic, and privacy concerns. We also identified a lack of standardized methods for evaluating LLMs’ effectiveness and feasibility. Conclusions and relevance This review thus highlights potential future directions and questions to address these limitations and to further explore LLMs’ potential in enhancing healthcare delivery.

Abstract Background Non-alcoholic fatty liver disease (NAFLD) describes a spectrum of chronic fattening of liver that can lead to fibrosis and cirrhosis. Diabetes has been identified as a major comorbidity that contributes to NAFLD progression. Health systems around the world make use of administrative data to conduct population-based prevalence studies. To that end, we sought to assess the accuracy of diabetes International Classification of Diseases (ICD) coding in administrative databases among a cohort of confirmed NAFLD patients in Calgary, Alberta, Canada. Methods The Calgary NAFLD Pathway Database was linked to the following databases: Physician Claims, Discharge Abstract Database, National Ambulatory Care Reporting System, Pharmaceutical Information Network database, Laboratory, and Electronic Medical Records. Hemoglobin A1c and diabetes medication details were used to classify diabetes groups into absent, prediabetes, meeting glycemic targets, and not meeting glycemic targets. The performance of ICD codes among these groups was compared to this standard. Within each group, the total numbers of true positives, false positives, false negatives, and true negatives were calculated. Descriptive statistics and bivariate analysis were conducted on identified covariates, including demographics and types of interacted physicians. Results A total of 12,012 NAFLD patients were registered through the Calgary NAFLD Pathway Database and 100% were successfully linked to the administrative databases. Overall, diabetes coding showed a sensitivity of 0.81 and a positive predictive value of 0.87. False negative rates in the absent and not meeting glycemic control groups were 4.5% and 6.4%, respectively, whereas the meeting glycemic control group had a 42.2% coding error. Visits to primary and outpatient services were associated with most encounters. Conclusion Diabetes ICD coding in administrative databases can accurately detect true diabetic cases. However, patients with diabetes who meets glycemic control targets are less likely to be coded in administrative databases. A detailed understanding of the clinical context will require additional data linkage from primary care settings.

At our centre, we introduced a continuous quality improvement (CQI) initiative during academic year 2018-19 targeting for repair multiple choice question (MCQ) items with discrimination index (D) < 0.1. The purpose of this study was to assess the impact of this initiative on reliability/internal consistency of our assessments. Our participants were medical students during academic years 2015-16 to 2020-21 and our data were summative MCQ assessments during this time. Since the goal was to systematically review and improve summative assessments in our undergraduate program on an ongoing basis, we used interrupted time series analysis to assess the impact on reliability. Between 2015-16 and 2017-18 there was a significant negative trend in the mean alpha coefficient for MCQ exams (regression coefficient -0.027 [-0.008, -0.047], p = 0.024). In the academic year following the introduction of our initiative (2018-19) there was a significant increase in the mean alpha coefficient (regression coefficient 0.113 [0.063, 0.163], p = 0.010) which was then followed by a significant positive post-intervention trend (regression coefficient 0.056 [0.037, 0.075], p = 0.006). In conclusion, our CQI intervention resulted in an immediate and progressive improvement reliability of our MCQ assessments.

BackgroundThe systematic review of clinical research papers is a labor-intensive and time-consuming process that often involves the screening of thousands of titles and abstracts. The accuracy and efficiency of this process are critical for the quality of the review and subsequent health care decisions. Traditional methods rely heavily on human reviewers, often requiring a significant investment of time and resources. ObjectiveThis study aims to assess the performance of the OpenAI generative pretrained transformer (GPT) and GPT-4 application programming interfaces (APIs) in accurately and efficiently identifying relevant titles and abstracts from real-world clinical review data sets and comparing their performance against ground truth labeling by 2 independent human reviewers. MethodsWe introduce a novel workflow using the Chat GPT and GPT-4 APIs for screening titles and abstracts in clinical reviews. A Python script was created to make calls to the API with the screening criteria in natural language and a corpus of title and abstract data sets filtered by a minimum of 2 human reviewers. We compared the performance of our model against human-reviewed papers across 6 review papers, screening over 24,000 titles and abstracts. ResultsOur results show an accuracy of 0.91, a macro F1-score of 0.60, a sensitivity of excluded papers of 0.91, and a sensitivity of included papers of 0.76. The interrater variability between 2 independent human screeners was κ=0.46, and the prevalence and bias-adjusted κ between our proposed methods and the consensus-based human decisions was κ=0.96. On a randomly selected subset of papers, the GPT models demonstrated the ability to provide reasoning for their decisions and corrected their initial decisions upon being asked to explain their reasoning for incorrect classifications. ConclusionsLarge language models have the potential to streamline the clinical review process, save valuable time and effort for researchers, and contribute to the overall quality of clinical reviews. By prioritizing the workflow and acting as an aid rather than a replacement for researchers and reviewers, models such as GPT-4 can enhance efficiency and lead to more accurate and reliable conclusions in medical research.

Abstract Background Parents of infants in neonatal intensive care units (NICUs) are often unintentionally marginalized in pursuit of optimal clinical care. Family Integrated Care (FICare) was developed to support families as part of their infants’ care team in level III NICUs. We adapted the model for level II NICUs in Alberta, Canada, and evaluated whether the new Alberta FICare™ model decreased hospital length of stay (LOS) in preterm infants without concomitant increases in readmissions and emergency department visits. Methods In this pragmatic cluster randomized controlled trial conducted between December 15, 2015 and July 28, 2018, 10 level II NICUs were randomized to provide Alberta FICare™ (n = 5) or standard care (n = 5). Alberta FICare™ is a psychoeducational intervention with 3 components: Relational Communication, Parent Education, and Parent Support. We enrolled mothers and their singleton or twin infants born between 32 0/7 and 34 6/7 weeks gestation. The primary outcome was infant hospital LOS. We used a linear regression model to conduct weighted site-level analysis comparing adjusted mean LOS between groups, accounting for site geographic area (urban/regional) and infant risk factors. Secondary outcomes included proportions of infants with readmissions and emergency department visits to 2 months corrected age, type of feeding at discharge, and maternal psychosocial distress and parenting self-efficacy at discharge. Results We enrolled 654 mothers and 765 infants (543 singletons/111 twin cases). Intention to treat analysis included 353 infants/308 mothers in the Alberta FICare™ group and 365 infants/306 mothers in the standard care group. The unadjusted difference between groups in infant hospital LOS (1.96 days) was not statistically significant. Accounting for site geographic area and infant risk factors, infant hospital LOS was 2.55 days shorter (95% CI, − 4.44 to − 0.66) in the Alberta FICare™ group than standard care group, P = .02. Secondary outcomes were not significantly different between groups. Conclusions Alberta FICare™ is effective in reducing preterm infant LOS in level II NICUs, without concomitant increases in readmissions or emergency department visits. A small number of sites in a single jurisdiction and select group infants limit generalizability of findings. Trial registration ClinicalTrials.gov Identifier NCT02879799 , retrospectively registered August 26, 2016.

Background: Despite their proven efficacy to reduce cardiovascular disease, statin medication use remains low in individuals at high risk of cardiovascular disease considering their widespread availability and safety. Our objective was to explore the perspectives of patients and family physicians with regard to the barriers and facilitators of statin use in primary care. Methods: In this qualitative descriptive study, we conducted 2 focus groups with patients (number, n = 8/6) and individual semistructured interviews with family physicians (n = 17) from community settings. Interviewers asked participants about barriers to and facilitators of statin use. Focus groups and interviews were digitally recorded, transcribed, and analyzed in duplicate using conventional content analysis. Results: Patients were averse to taking statins for a variety of reasons: medication avoidance and burden; inadequate buy-in for statin therapy; and difficulty remembering to take statins regularly. Family physicians perceived similar barriers and reported other barriers: lack of resources such as inadequate tracking systems; specialist-primary care provider guideline discordance; and lack of continuity and relationship. Patients expressed that key facilitators were patient education and support; splitting tablets to increase cost-effectiveness; and changing to a different statin or lower dose in those with side effects. Family physicians described several similar strategies to facilitate therapy as well as shared decision making and clinical decision support tools as enablers for improvement. Conclusions: We identified several important barriers to and facilitators of statin use at the patient and prescriber level. This information offers insight into strategies to improve statin use and the development of innovative programs and interventions. Résumé: Introduction: En dépit de leur efficacité prouvée pour réduire les maladies cardiovasculaires, l’utilisation des statines reste faible chez les individus exposés à un risque élevé de maladies cardiovasculaires si l’on considère leur grande disponibilité et leur innocuité. Notre objectif était d’examiner les perspectives des patients et des médecins de famille en ce qui concerne les obstacles et les facilitateurs de l’utilisation des statines en soins primaires. Méthodes: Dans la présente étude qualitative descriptive, nous avons mené 2 groupes de discussion composés de patients (nombre, n = 8/6) et des entrevues semi-structurées individuelles avec des médecins de famille (n = 17) en milieu communautaire. Les intervieweurs ont demandé aux participants quels étaient les obstacles et les facilitateurs de l’utilisation des statines. Les groupes de discussion et les entrevues étaient enregistrés numériquement, transcrits et analysés en duplicata à l’aide de l’analyse de contenu traditionnelle. Résultats: Les patients se sont opposés à la prise de statines pour plusieurs raisons : l’évitement et le fardeau des médicaments, l’adhésion insuffisante au traitement par statines et la difficulté à se souvenir de prendre régulièrement les statines. Les médecins de famille ont perçu des obstacles similaires et ont rapporté d’autres obstacles dont le manque de ressources telles que les systèmes de suivi inadéquats, la divergence entre les orientations des spécialistes et des prestataires de soins primaires, et le manque de continuité et de relation. Les patients ont exprimé que les principaux facilitateurs étaient l’éducation et le soutien offerts aux patients; le fractionnement des comprimés pour améliorer l’efficience; le changement vers une statine différente ou une dose plus faible chez ceux qui présentent des effets secondaires. Les médecins de famille ont décrit plusieurs stratégies semblables pour faciliter le traitement ainsi que la prise de décision partagée et les outils d’aide à la décision clinique qui facilitent l’amélioration. Conclusions: Nous avons déterminé plusieurs obstacles et facilitateurs importants de l’utilisation des statines au point de vue du patient et du prescripteur. Ces informations offrent un aperçu des stratégies pour améliorer l’utilisation des statines et l’élaboration d’interventions et de programmes innovateurs.

Objective To evaluate the barriers and facilitators to implementing Alberta Family Integrated Care (AB-FICare [2019 Benzies]), a model of care for integrating parents into level II neonatal intensive care units (NICUs) care teams, from the perspective of healthcare providers (HCP) and hospital administrators.Design Qualitative process evaluation substudy.Setting Ten level II NICUs in six cities across Alberta, Canada.Participants HCP and hospital administrators (n=32) who were involved in the cluster-randomised controlled trial of AB-FICare in level II NICUs.Methods Post-implementation semi-structured interviews were conducted via phone or in-person. The Consolidated Framework for Implementation Research was used to develop interview guides, code transcripts and analyse data.Results Key facilitators to implementation of AB-FICare included (1) a receptive implementation climate, (2) compatibility of the intervention with individual and organisational practices, (3) available resources and access to knowledge and information for HCP and hospital administrators, (4) engagement of key stakeholders across the organisation, (5) engagement of and outcomes for intervention participants, and (6) reflecting and evaluating on implementation progress and patient and family outcomes. Barriers were (1) design quality and packaging of the intervention, (2) relative priority of AB-FICare in relation to other initiatives, and (3) learning climate within the organisation. Mixed influences on implementation depending on contextual factors were coded to eight constructs: intervention source, cost, peer pressure, external policy and incentives, staff needs and resources, structural characteristics, organisational incentives and rewards, and knowledge, beliefs and attitudes.Conclusions The characteristics of an organisation and the implementation process had largely positive influences, which can be leveraged for implementation of AB-FICare in the NICU. We recommend site-specific consultations to mitigate barriers and assess how swing factors might impact implementation given the local context, with the goal that strategies can be put in place to manage their influence on implementation.Trial registration number NCT02879799.

Abstract Background The first Canadian outbreak of community-associated methicillin-resistant Staphylococcus aureus (CA-MRSA) was identified in 2004 in Calgary, Alberta. Using a novel model of MRSA population-based surveillance, sociodemographic risk associations, yearly geospatial dissemination and prevalence of CA-MRSA infections over an 11 year period was identified in an urban healthcare jurisdiction of Calgary. Methods Positive MRSA case records, patient demographics and laboratory data were obtained from a centralized Laboratory Information System of Calgary Laboratory Services in Calgary, Alberta, Canada between 2004 and 2014. Public census data was obtained from Statistics Canada, which was used to match with laboratory data and mapped using Geographic Information Systems. Results During the study period, 52.5% of positive MRSA infections in Calgary were CA-MRSA cases. The majority were CMRSA10 (USA300) clones (94.1%; n = 4255), while the remaining case (n = 266) were CMRSA7 (USA400) clones. Period prevalence of CMRSA10 increased from 3.6 cases/100000 population in 2004, to 41.3 cases/100000 population in 2014. Geospatial analysis demonstrated wide dissemination of CMRSA10 annually in the city. Those who are English speaking (RR = 0.05, p

Abstract Objective The incidence of the combined myeloproliferative neoplasms (MPNs) was determined for a major Canadian city. Retrospective cases of MPN diagnoses (essential thrombocythemia, polycythemia vera, and primary myelofibrosis) between 2011 to 2015 were retrieved from the Southern Alberta Cancer Cytogenetics Laboratory’s database at Alberta Public Laboratories. Results An incidence rate of 2.05 cases per 100,000 person-years (95% CI 1.73–2.41) was determined, giving an age-standardized Canadian incidence of 2.71 cases per 100,000 person years (95% CI 2.63–2.78). MPN diagnoses occurred at a wide age range of 8–93 (median 66) and an age-dependent increase in incidence. Incidence rates for the MPNs are first reported here for a Canadian population.

Objective The objective of our study was to explore the perspectives of patients and general practitioners (GPs) regarding interventions to increase initiation of cholesterol lowering medication (or statins), including a proposed laboratory-based facilitated relay intervention.Design Qualitative descriptive study using interviews and focus groups for data collection, and thematic analysis for data analysis.Setting Primary care providers and patients in Calgary, Alberta, Canada.Participants 17 GPs with primarily community-based, non-academic practices with at least 1 year of practice experience participated in semistructured interviews. 14 patients at high risk of cardiovascular disease participated in focus groups.Main outcome measures Exploration of strategies that might be used to enhance the prescription of, and adherence to statin therapy for patients with statin-indicated conditions.Results GPs proposed a variety of interventions to improve statin prescription, including electronic record audit solutions, GP directed education, and patient-oriented campaigns. Patients expressed that they may benefit from being provided access to their laboratory test results, as well as targeted education. Both parties provided positive feedback on the proposed laboratory-based facilitated relay intervention, while pointing out areas for improvement. Notably, GPs were concerned that the patient-directed component of the intervention might jeopardise therapeutic relationships, and patients were concerned about accidental disclosure of personal health information. Important considerations for the design of facilitated relay messaging should include brevity, simplicity and the provision of contact information for inquiries.Conclusions GPs and patients described several suggestions for increasing statin initiation and welcomed the proposal of a laboratory-based facilitated relay strategy. These findings support further testing of this intervention which may enhance GPs’ ability to successfully engage patients in cardiovascular risk reduction through statin therapy.

Data presented in this data article include the proportion and counts of community-based adult men and women who fasted for various lipid and diabetes screening-related tests from January 2010 to June 2016 in Calgary, Alberta, Canada. This data set was collected from Calgary Laboratory Service׳s Laboratory Information System, and included 3,003,667 testing encounters (1,447,720 testing encounters for men and 1,555,947 testing encounters for women) over the age of 18 years, who fasted for a lipid profile, lipid+random blood glucose (RBG), lipid+hemoglobin A1c (HbA1c), lipid+RBG+HbA1c, lipid+fasting blood glucose (FBG), lipid+FBG+RBG, lipid+FBG+HbA1c, or lipid+FBG+RBG+HbA1c. Data are related to “Proportion of adults fasting for lipid testing relative to guideline changes in Alberta” (Ma et al., 2017).

All laboratory tests performed within the province of Alberta in Canada are captured by three Laboratory Information Systems (LIS; Millennium, Sunquest and Meditech), which comprise the provincial Consolidated Laboratory Data Repository (CLDR). The following secondary laboratory data for electrolyte panel (EP) and complete blood count (CBC) test panels performed in emergency room (ER) and inpatient settings were collected from January 1 – December 31, 2018: total test panel volume, total number of test panels repeated, number of test panels repeated within the 24 hour period, test result, date of testing, time of test, and patient Provincial Health Number (PHN). Patient PHN were used as a linking variable to match subsequent tests performed on the same patient. The first time a test was recorded per patient was defined as the “index test”. If the same test panel was performed within a 24-h period following the index test for the patient, data for the repeated test panel was also collected. The index test was defined as “normal” or “abnormal” according to established laboratory normal values and laboratory test reference ranges. For CBCs, we considered the panel to be abnormal if any of the hematocrit (Hct), hemoglobin (Hgb), mean corpuscular hemoglobin concentration (MCHC), mean corpuscular volume (MCV), platelet (PLT), red blood cell (RBC), red cell distribution width (RDW) or white blood cell (WBC) values were outside the normal laboratory reference range. For electrolyte panels, we considered the panel to be abnormal if any of the chloride (Cl), potassium (K), and sodium (Na) were outside of the normal laboratory reference range. All EP results were from clinical chemistry analyzers only. The reuse potential of this dataset can allow other jurisdictions in Canada to compare their redundant repeat testing in their hospital settings with this dataset as a benchmark. This article was submitted via another Elsevier journal as a co-submission (“Inappropriate repeat testing of complete blood count (CBC) and electrolytes in inpatients from Alberta, Canada” [1]), and readers should refer to the co-submission article for interpretation of the results. Keywords: Laboratory utilization, Repeat testing, Complete blood count, Electrolyte panel, Laboratory medicine, Health informatics

Abstract Objective The epidemiology of chronic myeloid leukemia is shifting due to the aging global population and the recent discovery and availability of targeted treatment options. This study provides recent data regarding the incidence of CML in Calgary, a major Canadian city. Data from patients diagnosed with CML by bone marrow sample analysis from 2011 to 2015 were collected from the database of the sole centralized cytogenetics facility in service of Calgary and its surrounding area. Results With an average of 10.2 newly diagnosed cases per year in Calgary from 2011 to 2015, the incidence rate was calculated to be 0.75 cases per 100,000 person-years (95% CI 0.57–0.99). With age standardization, the incidence was 0.87 cases per 100,000 person-years (95% CI 0.82–0.91) for the Canadian population, which was low compared to other developed Western nations. The highest incidence rates were observed in the older patient categories, however there was a broad age distribution for incident cases and the median age at diagnosis was 48. There was a general male bias for CML most pronounced at the younger ages. Our description of CML incidence will help to inform healthcare planners amidst the dramatically altered treatment of this hematological neoplasm.

Abstract Background Chronic kidney disease (CKD) is often asymptomatic in its early stages but is indicated and is diagnosed with an estimated glomerular filtration rate (eGFR)

Abstract Objective Acute lymphocytic leukemia (ALL) is a rare malignant neoplasm that develops from abnormal lymphoid stem cells. ALL incidence is highest among children and declines towards adolescence. There is limited data on the epidemiology of ALL, especially in Canada. This retrospective cohort study used patient data from the Calgary Laboratory Services Cancer Cytogenetics Laboratory to report the incidence rate of ALL in Calgary, Alberta, Canada. New cases of ALL were identified for the 5-year period of January 1, 2011 until December 31, 2015. Reported incidence rates were categorized by sex and age groups, and age-standardized to the Canadian population. Results There were an average of 11.4 new cases of ALL diagnosed per year between 2011 and 2015. The total incidence rate per 100,000 person-years was 0.84. Incidence rates peaked in children aged 0–4 with 7.55 and 3.32 cases per 100,000 person-years for males and females, respectively. The median age of diagnosis was 8 years. Incidence rates were generally lowest for adults aged 20 and over. The ratio of males to females diagnosed with ALL was 1.59. Overall, the recent incidence of ALL in Calgary is comparatively low with a preference for males and children below 5 years of age.

Abstract Background Many clinical practice guidelines encourage diagnosis and empiric treatment of lower urinary tract infection without laboratory investigation; however, urine culture testing remains one of the largest volume tests in the clinical microbiology laboratory. In this study, we sought to determine if there were specific patient groups to which increased testing was directed. To do so, we combined laboratory data on testing rates with Census Canada sociodemographic data. Methods Urine culture testing data was obtained from the Calgary Laboratory Services information system for 2011. We examined all census dissemination areas within the city of Calgary and, for each area, testing rates were determined for age and gender cohorts. We then compared these testing rates to sociodemographic factors obtained from Census Canada and used Poisson regression and generalized estimating equations to test associations between testing rates and sociodemographic variables. Results Per capita urine culture testing is increasing in Calgary. For 2011, 100,901 individuals (9.2% of all people) received urine cultures and were included in this analysis. The majority of cultures were received from the community (67.9%). Substantial differences in rate of testing were observed across the city. Most notably, urine culture testing was drastically lower in areas of high (≥ $100000) household income (RR = 0.07, p

Abstract Background Every year, about 15 million of the world’s infants are born preterm (before 37 weeks gestation). In Alberta, the preterm birth rate was 8.7% in 2015, the second highest among Canadian provinces. Approximately 20% of preterm infants are born before 32 weeks gestation (early preterm), and require care in a Level III neonatal intensive care unit (NICU); 80% are born moderate (32 weeks and zero days [320/7] to 336/7 weeks) and late preterm (340/7 to 366/7 weeks), and require care in a Level II NICU. Preterm birth and experiences in the NICU disrupt early parent-infant relationships and induce parental psychosocial distress. Family Integrated Care (FICare) shows promise as a model of care in Level III NICUs. The purpose of this study is to evaluate length of stay, infant and maternal clinical outcomes, and costs following adaptation and implementation of FICare in Level II NICUs. Methods We will conduct a pragmatic, cluster randomized controlled trial (cRCT) in ten Alberta Level II NICUs allocated to one of two groups: FICare or standard care. The FICare Alberta model involves three theoretically-based, standardized components: information sharing, parenting education, and family support. Our sample size of 181 mother-infant dyads per group is based on the primary outcome of NICU length of stay, 80% participation, and 80% retention at follow-up. Secondary outcomes (e.g., infant clinical outcomes and maternal psychosocial distress) will be assessed shortly after admission to NICU, at discharge and 2 months corrected age. We will conduct economic analysis from two perspectives: the public healthcare payer and society. To understand the utility, acceptability, and impact of FICare, qualitative interviews will be conducted with a subset of mothers at the 2-month follow-up, and with hospital administrators and healthcare providers near the end of the study. Discussion Results of this pragmatic cRCT of FICare in Alberta Level II NICUs will inform policy decisions by providing evidence about the clinical effectiveness and costs of FICare. Trial registration ClinicalTrials.gov, ID: NCT02879799 . Registered on 27 May 2016. Protocol version: 9 June 2016; version 2.

Abstract Background The incidence rate of acute myeloid leukemia (AML) was determined in the Calgary Metropolitan Area, a major Canadian city. Methods Data from all patients diagnosed with AML between January 1, 2011 and December 31, 2015 were retrieved from a single, centralized cancer cytogenetics laboratory for bone marrow samples, the sole diagnostic facility of its kind in Southern Alberta. Results The calculated incidence rate was 2.79 cases per 100,000 person-years with a median age of 60, slightly lower than previously published data. The age-standardized incidence rate for Canada was 3.46 cases per 100,000 person-years. The higher value is reflective of Calgary’s younger population compared to the rest of Canada. Higher male incidence and greatest incidence occurring at approximately the age of 85 is similar to data from other developed countries. The lower incidence rates and median age of diagnosis, in comparison with that of other high-income nations, may be due to differences in the proportion of aging citizens in the population. Conclusion This is the first published incidence rate of acute myeloid leukemia (AML) in Canada across all age groups.

After publication of our article [1], the authors have reported mathematical errors made in the sample size calculation for this cluster randomized controlled trial (cRCT) (Benzies et al. 2017).

The data presented in this article is the provincial vitamin B12 test volume data for Alberta, Canada per month between April 1, 2015 and April 30, 2018. This data set was collected from the three different Alberta Public Laboratories Laboratory Information Systems: Cerner Millennium for Calgary, Sunquest for Edmonton, and MediTech for the remaining rural zones of Alberta (Bonnyville, Grand Prairie, Camrose, Red Deer, and Medicine Hat). An educational province-wide intervention aimed at reducing redundant testing was implemented on April 11, 2017 in Calgary, Alberta and Edmonton, Alberta and on May 2, 2017 in rural Alberta sites. All vitamin B12 test results in Alberta were appended with the educational comment “A normal test result indicates adequate stores and should not be repeated. However, if specific clinical situations require re-testing, the interval should not be sooner than 1 year.” Provincial monthly test volumes prior to this intervention ranged from 54,182 to 73,522 tests per month and after this intervention ranged from 59,116 to 74,006 tests per month. The total number of vitamin B12 tests ordered over the 37 months in Alberta was 2,444,724; 690,448 tests were ordered in Calgary, 1,029,315 tests were ordered in Edmonton, and 724,961 tests were ordered in rural sites. This data article was submitted as a companion paper to the related research article, “Implementation of an educational province-wide intervention to reduce redundant vitamin B12 testing: a cross-sectional study”[1]. Keywords: Vitamin B12, Laboratory utilization, Inappropriate testing, Laboratory medicine, Pathology informatics

This data incorporates 2016 testing volumes ordered by family physicians and performed at Calgary Laboratory Services (CLS), the sole supplier of clinical laboratory services for the catchment area of the City of Calgary, Alberta, Canada. For each test, the mean number of tests ordered per patient was calculated over ordering Calgary physicians, along with arithmetic coefficients of variation (CV's). The latter parameter is reflective of variance in ordering practice among family physicians practicing in Calgary and is proposed as a benchmark measure for laboratory utilization in our accompanying research article [1]. The data table encompasses 358 tests ordered by at least 3 family physicians at a minimum total frequency of 100 within the 2016 study period and is presented in ascending order of rank in CV. Keywords: Clinical laboratory tests, Utilization, Variance, Laboratory medicine, Family medicine

Data presented in this article include the top 51 ordered laboratory tests in Calgary and surrounding area, Alberta, from January to December 2017. This data set was collected from Calgary Laboratory Service’s Laboratory Information System, and included top 51 tests ordered from community (n = 11, 224, 330), inpatient (n = 2,340,594) and emergency (n = 1,670,062) settings. Test order mnemonic that were not true laboratory tests (eg: “extra PST tube”, “extra tube”, etc.) were excluded in the analysis. The top test ordered in all 3 test encounters was the complete blood count test (community encounter, n = 921, 873; inpatient setting, n = 357, 375; and emergency setting, n = 276, 954). This data article was submitted as a companion paper to the related research article, “Estimated costs of 51 commonly ordered laboratory tests in Canada” [1]. Keywords: Laboratory medicine, Pathology informatics, Laboratory utilization management

Polypharmacy with “potentially inappropriate medications” (PIMs) and “potential prescribing omissions” (PPOs) are frequent among those 65 and older. We assessed PIMs and PPOs in a retrospective study of 82,935 patients ≥ 65 during their first admission in the period March 2013 through February 2018 to the four acute-care Calgary hospitals. We used the American Geriatric Society (AGS) and STOPP/START criteria to assess PIMs and PPOs. We computed odds ratios (ORs) for key outcomes of concern to patients, their families, and physicians, namely readmission and/or mortality within six months of discharge, and controlled for age, sex, numbers of medications, PIMs, and PPOs. For readmission, the adjusted OR for number of medications was 1.09 (1.09–1.09), for AGS PIMs 1.14 (1.13–1.14), for STOPP PIMs 1.15 (1.14–1.15), for START PPOs 1.04 (1.02–1.06), and for START PPOs correctly prescribed 1.16 (1.14–1.17). For mortality within 6 months of discharge, the adjusted OR for the number of medications was 1.02 (1.01–1.02), for STOPP PIMs 1.07 (1.06–1.08), for AGS PIMs 1.11 (1.10–1.12), for START PPOs 1.31 (1.27–1.34), and for START PPOs correctly prescribed 0.97 (0.94–0.99). Algorithm rule mining identified an 8.772 higher likelihood of mortality with the combination of STOPP medications of duplicate drugs from the same class, neuroleptics, and strong opioids compared to a random relationship, and a 2.358 higher likelihood of readmission for this same set of medications. Detailed discussions between patients, physicians, and pharmacists are needed to improve these outcomes.

ImportanceHigher levels of red cell distribution width (RDW) are associated with adverse outcomes, especially in selected cohorts with or at risk for chronic disease. Whether higher RDW or the related parameter standard deviation of the red blood cell distribution (SD-RBC) can predict a broader range of outcomes in the general population is unknown.ObjectiveTo evaluate the association of RDW and SD-RBC with the risk of adverse outcomes in people from the general population.DesignPopulation-based retrospective cohort study.SettingHealth care system in a Canadian province (Alberta).ParticipantsAll 3,156,863 adults living in Alberta, Canada with at least one measure of RDW and SD-RBC between 2003 and 2016. Data were analyzed in September 2018.ExposureRDW and SD-RBC, classified into percentiles (99).Main outcomesAll-cause death, first myocardial infarction, first stroke or transient ischemic attack, placement into long-term care (LTC), progression to renal replacement therapy (initiation of chronic dialysis or pre-emptive kidney transplantation), incident solid malignancy, and first hospitalization during follow-up.ResultsOver median follow-up of 6.8 years, 209,991 of 3,156,863 participants (6.7%) died. The risk of death increased with increasing RDW percentile. After adjustment, and compared to RDW in the 25th to 75th percentiles, the risk of death was lower for participants in the 99th percentile (HR 2.18, 95% CI 2.12,2.23). Similar results were observed for MI, stroke/TIA, incident cancer, hospitalization and LTC placement, but no association was found between RDW and ESRD. Findings were generally similar for SD-RBC, except that all associations tended to be stronger than for RDW, and both lower and higher values of SD-RBC were independently associated with ESRD.Conclusion and relevanceRDW and SD-RBC may be useful as prognostic markers for people in the general population, especially for outcomes related to chronic illness. SD-RBC may be superior to RDW.

Introduction While the effectiveness of cervical cancer-screening (CCS) programs is well accepted, concern is growing regarding Family physicians (FP) poor adherence to CCS guidelines resulting in over and under screening. In Canada, it is a FP responsibility to ensure that the CCS is done as per guidelines and with appropriate follow-up. Objectives and Approach To identify primary care physicians’ characteristics that are associated with over and under CCS for eligible women in Calgary, Alberta. We accessed the Calgary Laboratory Services data for 1475 FPs practicing in Calgary and linked it with the Physicians database of College of Physicians and Surgeons Alberta database. We then matched FP’s gender, country and year of medical school graduation, years since medical school graduation, certification in family medicine and their clinic address with their CCS testing patterns. Using doctors as their own controls, we compared data from 2010-2016 to determine practice variations in CCS patterns subsequent to guideline changes. Results We analyzed approximately 2,400,000 Pap test requisitions (approx. 300,000 per year) to identify screening patterns from 2010-2016 of 1475 family practitioners practicing in Calgary. Our preliminary results identified significant variations in the test ordering patterns of FPs. Approx. half of the male FPs were not performing CCS tests on their eligible female patients. Female FPs ordered more CCS tests than their male counterparts. FP trained in North America, were ordering more pap tests than FPS trained elsewhere. Decreased CSS was also observed among FPs practicing in Northeast Calgary. Conclusion/Implications We detected three CCS patterns: FPs who never perform CSS on eligible female patients; FPs who followed recommended guidelines for performing CCS tests and FPs who performed CCS tests, not following the guidelines. To ensure appropriate use of CSS, identifying intention-behavior relationships and innovative educational interventions for FPs are required.

Introduction Cervical Cancer Screening (CCS) has reduced the incidence and mortality rates of cervical cancer (CC). However, the benefits are distributed unevenly since 30% of eligible women have not been screened within three years in Alberta. Women who have never been screened or are screened irregularly are most at risk for CC. Objectives and Approach The aim of this study was to understand who gets CCS and who does not, in Calgary, Alberta and analyze the CC policy implications since 2006-2016. CCS information of women aged 25-69 were obtained from Calgary Laboratory Services for the years 2006, 2011 and 2016 and matched with Canadian Census data. Negative binomial regression and Generalized Estimating Equations were used to test associations of CCS rates with socio-demographic variables for eligible women. CCS spatial trends over the years was studied using the GIS Hotspot analysis. Results Major age and geographical variations were observed in CCS rates in Calgary. CCS rates in the recommended age groups varied from 40.6 % to 23.6 %. For age groups between 25 and 54, CCS rates were above 33\%, which implies that many women are having tests more than once every three years. Use was positively associated with median household income, education, Chinese ethnicity and negatively associated with ‘Black’ visible minority status. Women living in lower socio-economic areas of Calgary are screened at lower rates. Hotspot analysis maps revealed heterogeneous testing patterns in the city with relatively higher testing in the downtown, Southeast and Northwest quadrants of the city and relatively decreased CCS in the Northeast quadrant of Calgary Conclusion/Implications Screening programs need to be strengthened with greater focus on including specific demographic groups and reducing overuse. Understanding current testing patterns are important in assessing the benefit to harm ratio of CCS and for monitoring and evaluation of CCS program.

To determine and quantify the prevalence of patient, provider and system level barriers to achieving diabetes care goals; and to examine whether barriers were different for people with poor glycemic control (HbA1c ≥ 10%; 86 mmol/mol) compared to fair glycemic control (7 to

BackgroundVitamin D deficiency is associated with an increased risk of cardiovascular disease; however, it is unclear whether vitamin D status should be considered in clinical risk assessments of patients with cardiovascular disease. Methods and ResultsThis study included 2975 patients who had their first serum total 25‐hydroxy vitamin D (25‐OH vitamin D) measurement before their first coronary catheterization in Alberta, Canada. Cox regression was used to examine associations between 25‐OH vitamin D and mortality risk after adjusting for demographic and clinical risk factors. Interactions were tested using multiplicative terms, and prognostic value was assessed using measures of model discrimination, fit, calibration and net reclassification improvement. There were 401 deaths over a median of 5.8 years of follow‐up. Serum total 25‐OH vitamin D was inversely associated with mortality after adjusting for demographic and clinical risk factors, which was largely driven by excess risk in the bottom quintile (hazard ratio 1.84 for bottom versus top quintile, 95% CI 1.36–2.50, P for trend

The purpose is to systematically review randomised controlled trials (RCTs) to change family physicians’ laboratory test-ordering. We searched 15 electronic databases (no language/date limitations). We identified 29 RCTs (4,111 physicians, 175,563 patients). Six studies specifically focused on reducing unnecessary tests, 23 on increasing screening tests. Using Cochrane methodology 48.5% of studies were low risk-of-bias for randomisation, 7% concealment of randomisation, 17% blinding of participants/personnel, 21% blinding outcome assessors, 27.5% attrition, 93% selective reporting. Only six studies were low risk for both randomisation and attrition. Twelve studies performed a power computation, three an intention-to-treat analysis and 13 statistically controlled clustering. Unweighted averages were computed to compare intervention/control groups for tests assessed by >5 studies. The results were that fourteen studies assessed lipids (average 10% more tests than control), 14 diabetes (average 8% > control), 5 cervical smears, 2 INR, one each thyroid, fecal occult-blood, cotinine, throat-swabs, testing after prescribing, and urine-cultures. Six studies aimed to decrease test groups (average decrease 18%), and two to increase test groups. Intervention strategies: one study used education (no change): two feedback (one 5% increase, one 27% desired decrease); eight education + feedback (average increase in desired direction >control 4.9%), ten system change (average increase 14.9%), one system change + feedback (increases 5-44%), three education + system change (average increase 6%), three education + system change + feedback (average 7.7% increase), one delayed testing. The conclusions are that only six RCTs were assessed at low risk of bias from both randomisation and attrition. Nevertheless, despite methodological shortcomings studies that found large changes (e.g. >20%) probably obtained real change.

Background: Demand forecasting is the area of predictive analytics devoted to predicting future volumes of services or consumables. Fair understanding and estimation of how demand will vary facilitates the optimal utilization of resources. In a medical laboratory, accurate forecasting of future demand, that is, test volumes, can increase efficiency and facilitate long-term laboratory planning. Importantly, in an era of utilization management initiatives, accurately predicted volumes compared to the realized test volumes can form a precise way to evaluate utilization management initiatives. Laboratory test volumes are often highly amenable to forecasting by time-series models; however, the statistical software needed to do this is generally either expensive or highly technical. Method: In this paper, we describe an open-source web-based software tool for time-series forecasting and explain how to use it as a demand forecasting tool in clinical laboratories to estimate test volumes. Results: This tool has three different models, that is, Holt-Winters multiplicative, Holt-Winters additive, and simple linear regression. Moreover, these models are ranked and the best one is highlighted. Conclusion: This tool will allow anyone with historic test volume data to model future demand.

Background: Red blood cell (RBC) analysis is a key feature in the evaluation of hematological disorders. The gold standard light microscopy technique has high sensitivity, but is a relativity time-consuming and labor intensive procedure. This study tested the sensitivity and specificity of gold standard light microscopy manual differential to the CellaVision ® DM96 (CCS; CellaVision, Lund, Sweden) automated image analysis system, which takes digital images of samples at high magnification and compares these images with an artificial neural network based on a database of cells and preclassified according to RBC morphology. Methods: In this study, 212 abnormal peripheral blood smears within the Calgary Laboratory Services network of hospital laboratories were selected and assessed for 15 different RBC morphologic abnormalities by manual microscopy. The same samples were reassessed as a manual addition from the instrument screen using the CellaVision ® DM96 system with 8 microscope high power fields (×100 objective and a 22 mm ocular). The results of the investigation were then used to calculate the sensitivity and specificity of the CellaVision ® DM96 system in reference to light microscopy. Results: The sensitivity ranged from a low of 33% (RBC agglutination) to a high of 100% (sickle cells, stomatocytes). The remainder of the RBC abnormalities tested somewhere between these two extremes. The specificity ranged from 84% (schistocytes) to 99.5% (sickle cells, stomatocytes). Conclusions: Our results showed generally high specificities but variable sensitivities for RBC morphologic abnormalities.

Context: Many hematology laboratories have adopted semi-automated digital platforms for routine use and the evidence supporting their use is increasing. Aims: The CellaVision platforms are among the most thoroughly studied digital hematology platforms; we wished to determine the accuracy of CellaVision for reticulocyte counting. Design, Materials and Methods: We compared reticulocyte counts performed manually, using the Beckman Coulter LH750 automated analyzer and with the CellaVision DM96 platform. We analyzed the results for pair-wise correlation and bias, and precision. Statistical Analyses Used: Analyses were performed using Statistical Package for the Social Sciences software (SPSS), including Spearman′s rho correlation coefficient, Friedman′s two-way Analysis Of Variance (ANOVA) for comparison of distributions; bias was compared by way of mean and standard deviation. Results: The CellaVision reticulocyte counts correlated most strongly with those of the analyzer (often considered the benchmark test); the reticulocyte count distributions were noted not to be significantly different from each other across all three methods. The mean and standard deviation of bias were lowest in the comparison of CellaVision and LH750 counts. Conclusions: Our data provide additional support for the accuracy of digital hematology applications using the CellaVision DM96 platform.

Peripheral blood smear image examination is a part of the routine work of every laboratory. The manual examination of these images is tedious, time-consuming and suffers from interobserver variation. This has motivated researchers to develop different algorithms and methods to automate peripheral blood smear image analysis. Image analysis itself consists of a sequence of steps consisting of image segmentation, features extraction and selection and pattern classification. The image segmentation step addresses the problem of extraction of the object or region of interest from the complicated peripheral blood smear image. Support vector machine (SVM) and artificial neural networks (ANNs) are two common approaches to image segmentation. Features extraction and selection aims to derive descriptive characteristics of the extracted object, which are similar within the same object class and different between different objects. This will facilitate the last step of the image analysis process: pattern classification. The goal of pattern classification is to assign a class to the selected features from a group of known classes. There are two types of classifier learning algorithms: supervised and unsupervised. Supervised learning algorithms predict the class of the object under test using training data of known classes. The training data have a predefined label for every class and the learning algorithm can utilize this data to predict the class of a test object. Unsupervised learning algorithms use unlabeled training data and divide them into groups using similarity measurements. Unsupervised learning algorithms predict the group to which a new test object belong to, based on the training data without giving an explicit class to that object. ANN, SVM, decision tree and K-nearest neighbor are possible approaches to classification algorithms. Increased discrimination may be obtained by combining several classifiers together.

Pathology informatics has evolved to varying levels around the world. The history of pathology informatics in different countries is a tale with many dimensions. At first glance, it is the familiar story of individuals solving problems that arise in their clinical practice to enhance efficiency, better manage (e.g., digitize) laboratory information, as well as exploit emerging information technologies. Under the surface, however, lie powerful resource, regulatory, and societal forces that helped shape our discipline into what it is today. In this monograph, for the first time in the history of our discipline, we collectively perform a global review of the field of pathology informatics. In doing so, we illustrate how general far-reaching trends such as the advent of computers, the Internet and digital imaging have affected pathology informatics in the world at large. Major drivers in the field included the need for pathologists to comply with national standards for health information technology and telepathology applications to meet the scarcity of pathology services and trained people in certain countries. Following trials by a multitude of investigators, not all of them successful, it is apparent that innovation alone did not assure the success of many informatics tools and solutions. Common, ongoing barriers to the widespread adoption of informatics devices include poor information technology infrastructure in undeveloped areas, the cost of technology, and regulatory issues. This review offers a deeper understanding of how pathology informatics historically developed and provides insights into what the promising future might hold.

Background: Leukocyte differentials are an important component of clinical care. Morphologic assessment of peripheral blood smears (PBS) may be required to accurately classify leukocytes. However, manual microscopy is labor intensive. The CellaVision DM96 is an automated system that acquires digital images of leukocytes on PBS, pre-classifies the cell type, and displays them on screen for a Technologist or Pathologist to approve or reclassify. Our study compares the results of the DM96 with manual microscopy. Methods: Three hundred and fifty-nine PBS were selected and assessed by manual microscopy with a 200 leukocyte cell count. They were then reassessed using the CellaVision DM96 with a 115 leukocyte cell count including reclassification when necessary. Correlation between the manual microscopy results and the CellaVision DM96 results was calculated for each cell type. Results: The correlation coefficients (r2 ) range from a high of 0.99 for blasts to a low of 0.72 for metamyelocytes. Conclusions: The correlation between the CellaVision DM96 and manual microscopy was as good or better than the previously published data. The accuracy of leukocyte classification depended on the cell type, and in general, there was lower correlation for rare cell types. However, the correlation is similar to previous studies on the correlation of manual microscopy with an established reference result. Therefore, the CellaVision DM96 is appropriate for clinical implementation.

Introduction: Rapid and accurate determination of platelet count is an important factor in diagnostic medicine. Traditional microscopic methods are labor intensive with variable results and are highly dependent on the individual training. Recent developments in automated peripheral blood differentials using a computerized system have shown many advantages as a viable alternative. The purpose of this paper was to determine the reliability and accuracy of the CellaVision DM 96 system with regards to platelet counts. Materials and Methods: One hundred twenty seven peripheral blood smears were analyzed for platelet count by manual microscopy, an automated hematology analyzer (Beckman Counter LH 780 or Unicel DXH 800 analyzers) and with the CellaVision DM96 system. Results were compared using the correlations and Bland-Altman plots. Results: Platelet counts from the DM96 system showed an R 2 of 0.94 when compared to manual platelet estimates and an R 2 of 0.92 when compared to the automated hematology analyzer results. Bland-Altman plots did not show any systematic bias.

Digital pathology is a rapidly evolving niche in the world of pathology and is likely to increase in popularity as technology improves. We performed a questionnaire for pathologists and pathology residents across Canada, in order to determine their current experiences and attitudes towards digital pathology; which modalities digital pathology is best suited for; and to assess the need for training in digital pathology amongst pathology residents and staff. An online survey consisting of 24 yes/no, multiple choice and free text questions regarding digital pathology was sent out via E-mail to all members of the Canadian Association of Pathologists and pathology residents across Canada. Survey results showed that telepathology (TP) is used in approximately 43% of institutions, primarily for teaching purposes (65%), followed by operating room consults (46%). Seventy-one percent of respondents believe there is a need for TP in their practice; 85% use digital images in their practice. The top two favored applications for digital pathology are teaching and consultation services, with the main advantage being easier access to cases. The main limitations of using digital pathology are cost and image/diagnostic quality. Sixty-two percent of respondents would attend training courses in pathology informatics and 91% think informatics should be part of residency training. The results of the survey indicate that Pathologists and residents across Canada do see a need for TP and the use of digital images in their daily practice. Integration of an informatics component into resident training programs and courses for staff Pathologists would be welcomed.

Introduction: Delta checks use two specimen test results taken in succession in order to detect test result changes greater than expected physiological variation. One of the most common and serious errors detected by delta checks is specimen mix-up errors. The positive and negative predictive values of delta checks for detecting specimen mix-up errors, however, are largely unknown. Materials and Methods: We addressed this question by first constructing a stochastic dynamic model using repeat test values for five analytes from approximately 8000 inpatients in Calgary, Alberta, Canada. The analytes examined were sodium, potassium, chloride, bicarbonate, and creatinine. The model simulated specimen mix-up errors by randomly switching a set number of pairs of second test results. Sensitivities and specificities were then calculated for each analyte for six combinations of delta check equations and cut-off values from the published literature. Results: Delta check specificities obtained from this model ranged from 50% to 99%; however the sensitivities were generally below 20% with the exception of creatinine for which the best performing delta check had a sensitivity of 82.8%. Within a plausible incidence range of specimen mix-ups the positive predictive values of even the best performing delta check equation and analyte became negligible. Conclusion: This finding casts doubt on the ongoing clinical utility of delta checks in the setting of low rates of specimen mix-ups.

Background: The use of adjuvant tamoxifen therapy in the treatment of estrogen receptor (ER) expressing breast carcinomas represents a major advance in personalized cancer treatment. Because there is no benefit (and indeed there is increased morbidity and mortality) associated with the use of tamoxifen therapy in ER-negative breast cancer, its use is restricted to women with ER expressing cancers. However, correctly classifying cancers as ER positive or negative has been challenging given the high reported false negative test rates for ER expression in surgical specimens. In this paper I model practice recommendations using published information from clinical trials to address the question of whether there is a false negative test rate above which it is more efficacious to forgo ER testing and instead treat all patients with tamoxifen regardless of ER test results. Methods: I used data from randomized clinical trials to model two different hypothetical treatment strategies: (1) the current strategy of treating only ER positive women with tamoxifen and (2) an alternative strategy where all women are treated with tamoxifen regardless of ER test results. The variables used in the model are literature-derived survival rates of the different combinations of ER positivity and treatment with tamoxifen, varying true ER positivity rates and varying false negative ER testing rates. The outcome variable was hypothetical 10-year survival. Results: The model predicted that there will be a range of true ER rates and false negative test rates above which it would be more efficacious to treat all women with breast cancer with tamoxifen and forgo ER testing. This situation occurred with high true positive ER rates and false negative ER test rates in the range of 20-30%. Conclusions: It is hoped that this model will provide an example of the potential importance of diagnostic error on clinical outcomes and furthermore will give an example of how the effect of that error could be modeled using real-world data from clinical trials.

Increased type I error resulting from multiple statistical comparisons remains a common problem in the scientific literature. This may result in the reporting and promulgation of spurious findings. One approach to this problem is to correct groups of P-values for "family-wide significance" using a Bonferroni correction or the less conservative Bonferroni-Holm correction or to correct for the "false discovery rate" with a Benjamini-Hochberg correction. Although several solutions are available for performing this correction through commercially available software there are no widely available easy to use open source programs to perform these calculations. In this paper we present an open source program written in Python 3.2 that performs calculations for standard Bonferroni, Bonferroni-Holm and Benjamini-Hochberg corrections.