Search

Background: Global pandemics like COVID-19 put a high amount of strain on health care systems and health workers worldwide. These crises generate a vast amount of news information published online across the globe. This extensive corpus of articles has the potential to provide valuable insights into the nature of ongoing events and guide interventions and policies. However, the sheer volume of information is beyond the capacity of human experts to process and analyze effectively., Objective: The aim of this study was to explore how natural language processing (NLP) can be leveraged to build a system that allows for quick analysis of a high volume of news articles. Along with this, the objective was to create a workflow comprising human-computer symbiosis to derive valuable insights to support health workforce strategic policy dialogue, advocacy, and decision-making., Methods: We conducted a review of open-source news coverage from January 2020 to June 2022 on COVID-19 and its impacts on the health workforce from the World Health Organization (WHO) Epidemic Intelligence from Open Sources (EIOS) by synergizing NLP models, including classification and extractive summarization, and human-generated analyses. Our DeepCovid system was trained on 2.8 million news articles in English from more than 3000 internet sources across hundreds of jurisdictions., Results: Rules-based classification with hand-designed rules narrowed the data set to 8508 articles with high relevancy confirmed in the human-led evaluation. DeepCovid's automated information targeting component reached a very strong binary classification performance of 98.98 for the area under the receiver operating characteristic curve (ROC-AUC) and 47.21 for the area under the precision recall curve (PR-AUC). Its information extraction component attained good performance in automatic extractive summarization with a mean Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score of 47.76. DeepCovid's final summaries were used by human experts to write reports on the COVID-19 pandemic., Conclusions: It is feasible to synergize high-performing NLP models and human-generated analyses to benefit open-source health workforce intelligence. The DeepCovid approach can contribute to an agile and timely global view, providing complementary information to scientific literature., (©Mathieu Ravaut, Ruochen Zhao, Duy Phung, Vicky Mengqi Qin, Dusan Milovanovic, Anita Pienkowska, Iva Bojic, Josip Car, Shafiq Joty. Originally published in JMIR AI (https://ai.jmir.org), 30.10.2024.)

Background: To investigate the impacts of the COVID-19 pandemic on the health workforce, we aimed to develop a framework that synergizes natural language processing (NLP) techniques and human-generated analysis to reduce, organize, classify, and analyze a vast volume of publicly available news articles to complement scientific literature and support strategic policy dialogue, advocacy, and decision-making., Objective: This study aimed to explore the possibility of systematically scanning intelligence from media that are usually not captured or best gathered through structured academic channels and inform on the impacts of the COVID-19 pandemic on the health workforce, contributing factors to the pervasiveness of the impacts, and policy responses, as depicted in publicly available news articles. Our focus was to investigate the impacts of the COVID-19 pandemic and, concurrently, assess the feasibility of gathering health workforce insights from open sources rapidly., Methods: We conducted an NLP-assisted media content analysis of open-source news coverage on the COVID-19 pandemic published between January 2020 and June 2022. A data set of 3,299,158 English news articles on the COVID-19 pandemic was extracted from the World Health Organization Epidemic Intelligence through Open Sources (EIOS) system. The data preparation phase included developing rules-based classification, fine-tuning an NLP summarization model, and further data processing. Following relevancy evaluation, a deductive-inductive approach was used for the analysis of the summarizations. This included data extraction, inductive coding, and theme grouping., Results: After processing and classifying the initial data set comprising 3,299,158 news articles and reports, a data set of 5131 articles with 3,007,693 words was devised. The NLP summarization model allowed for a reduction in the length of each article resulting in 496,209 words that facilitated agile analysis performed by humans. Media content analysis yielded results in 3 sections: areas of COVID-19 impacts and their pervasiveness, contributing factors to COVID-19-related impacts, and responses to the impacts. The results suggest that insufficient remuneration and compensation packages have been key disruptors for the health workforce during the COVID-19 pandemic, leading to industrial actions and mental health burdens. Shortages of personal protective equipment and occupational risks have increased infection and death risks, particularly at the pandemic's onset. Workload and staff shortages became a growing disruption as the pandemic progressed., Conclusions: This study demonstrates the capacity of artificial intelligence-assisted media content analysis applied to open-source news articles and reports concerning the health workforce. Adequate remuneration packages and personal protective equipment supplies should be prioritized as preventive measures to reduce the initial impact of future pandemics on the health workforce. Interventions aimed at lessening the emotional toll and workload need to be formulated as a part of reactive measures, enhancing the efficiency and maintainability of health delivery during a pandemic., (©Anita Pienkowska, Mathieu Ravaut, Maleyka Mammadova, Chin-Siang Ang, Hanyu Wang, Qi Chwen Ong, Iva Bojic, Vicky Mengqi Qin, Dewan Md Sumsuzzman, Onyema Ajuebor, Mathieu Boniol, Juana Paola Bustamante, James Campbell, Giorgio Cometto, Siobhan Fitzpatrick, Catherine Kane, Shafiq Joty, Josip Car. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.06.2024.)

Importance: Systems-level barriers to diabetes care could be improved with population health planning tools that accurately discriminate between high- and low-risk groups to guide investments and targeted interventions., Objective: To develop and validate a population-level machine learning model for predicting type 2 diabetes 5 years before diabetes onset using administrative health data., Design, Setting, and Participants: This decision analytical model study used linked administrative health data from the diverse, single-payer health system in Ontario, Canada, between January 1, 2006, and December 31, 2016. A gradient boosting decision tree model was trained on data from 1 657 395 patients, validated on 243 442 patients, and tested on 236 506 patients. Costs associated with each patient were estimated using a validated costing algorithm. Data were analyzed from January 1, 2006, to December 31, 2016., Exposures: A random sample of 2 137 343 residents of Ontario without type 2 diabetes was obtained at study start time. More than 300 features from data sets capturing demographic information, laboratory measurements, drug benefits, health care system interactions, social determinants of health, and ambulatory care and hospitalization records were compiled over 2-year patient medical histories to generate quarterly predictions., Main Outcomes and Measures: Discrimination was assessed using the area under the receiver operating characteristic curve statistic, and calibration was assessed visually using calibration plots. Feature contribution was assessed with Shapley values. Costs were estimated in 2020 US dollars., Results: This study trained a gradient boosting decision tree model on data from 1 657 395 patients (12 900 257 instances; 6 666 662 women [51.7%]). The developed model achieved a test area under the curve of 80.26 (range, 80.21-80.29), demonstrated good calibration, and was robust to sex, immigration status, area-level marginalization with regard to material deprivation and race/ethnicity, and low contact with the health care system. The top 5% of patients predicted as high risk by the model represented 26% of the total annual diabetes cost in Ontario., Conclusions and Relevance: In this decision analytical model study, a machine learning model approach accurately predicted the incidence of diabetes in the population using routinely collected health administrative data. These results suggest that the model could be used to inform decision-making for population health planning and diabetes prevention.

Across jurisdictions, government and health insurance providers hold a large amount of data from patient interactions with the healthcare system. We aimed to develop a machine learning-based model for predicting adverse outcomes due to diabetes complications using administrative health data from the single-payer health system in Ontario, Canada. A Gradient Boosting Decision Tree model was trained on data from 1,029,366 patients, validated on 272,864 patients, and tested on 265,406 patients. Discrimination was assessed using the AUC statistic and calibration was assessed visually using calibration plots overall and across population subgroups. Our model predicting three-year risk of adverse outcomes due to diabetes complications (hyper/hypoglycemia, tissue infection, retinopathy, cardiovascular events, amputation) included 700 features from multiple diverse data sources and had strong discrimination (average test AUC = 77.7, range 77.7-77.9). Through the design and validation of a high-performance model to predict diabetes complications adverse outcomes at the population level, we demonstrate the potential of machine learning and administrative health data to inform health planning and healthcare resource allocation for diabetes management.