Search

The first patient was misclassified in the diagnostic conclusion according to a local clinical expert opinion in a new clinical implementation of a knee osteoarthritis artificial intelligence (AI) algorithm at Bispebjerg-Frederiksberg University Hospital, Copenhagen, Denmark. In preparation for the evaluation of the AI algorithm, the implementation team collaborated with internal and external partners to plan workflows, and the algorithm was externally validated. After the misclassification, the team was left wondering: what is an acceptable error rate for a low-risk AI diagnostic algorithm? A survey among employees at the Department of Radiology showed significantly lower acceptable error rates for AI (6.8 %) than humans (11.3 %). A general mistrust of AI could cause the discrepancy in acceptable errors. AI may have the disadvantage of limited social capital and likeability compared to human co-workers, and therefore, less potential for forgiveness. Future AI development and implementation require further investigation of the fear of AI’s unknown errors to enhance the trustworthiness of perceiving AI as a co-worker. Benchmark tools, transparency, and explainability are also needed to evaluate AI algorithms in clinical implementations to ensure acceptable performance.

Rational and Objectives: This study investigated how an AI tool impacted radiologists reading time for non-contrast chest CT exams. Materials and Methods: An AI tool was implemented into the PACS reading workflow of non-contrast chest CT exams between April and May 2020. The reading time was recorded for one CONSULTANT RADIOLOGIST and one RADIOLOGY RESIDENT by an external observer. After each case radiologists answered questions regarding additional findings and perceived case overview. Reading times were recorded for 25 cases without and 20 cases with AI tool assistance for each reader. Differences in reading time with and without the AI tool were assessed using Welch's t-test for non-inferiority with non-inferiority limits defined as 100 seconds for the consultant and 200 seconds for the resident. Results: The mean reading time for the radiology resident was not significantly affected by the AI tool (without AI 370s vs with AI 437s; +67s 95% CI -28s to +163s, p = 0.16). The reading time for the radiology consultant was also not significantly affected by the AI tool (without AI 366s vs with AI 380s; +13s (95% CI - -57s to 84s, p = 0.70). The AI tool led to additional actionable findings in 5/40 (12.5%) studies and better overview in 18/20 (90%) of studies for the resident. Conclusion: A PACS based implementation of an AI tool for concurrent reading of chest CT exams did not increase reading time with additional actionable findings made as well as a perceived better case overview for the radiology resident.

Several large trials have evaluated the effect of CT screening based on specific symptoms, with varying outcomes. Screening of patients with CT based on their prognosis alone has not been examined before. For moderate-to-high risk patients presenting in the emergency department (ED), the potential gain from a CT scan might outweigh the risk of radiation exposure. We hypothesized that an accelerated “multiple rule out” CT screening of moderate-to-high risk patients will detect many clinically unrecognized diagnoses that affect change in treatment. Patients ≥ 40 years, triaged as high-risk or moderate-to-high risk according to vital signs, were eligible for inclusion. Patients were scanned with a combined ECG-gated and dual energy CT scan of cerebrum, thorax, and abdomen. The impact of the CT scan on patient diagnosis and treatment was examined prospectively by an expert panel. A total of 100 patients were included in the study, (53% female, mean age 73 years [age range, 43–93]). The scan lead to change in treatment or additional examinations in 37 (37%) patients, of which 24 (24%) were diagnostically significant, change in acute treatment in 11 (11%) cases and previously unrecognized malignant tumors in 10 (10%) cases. The mean size specific radiation dose was 15.9 mSv (± 3.1 mSv). Screening with a multi-rule out CT scan of high-risk patients in an ED is feasible and result in discovery of clinically unrecognized diagnoses and malignant tumors, but at the cost of radiation exposure and downstream examinations. The clinical impact of these findings should be evaluated in a larger randomized cohort.