Search

Despite frequent reports of imaging artificial intelligence (AI) that parallels human performance, clinicians often question the safety and robustness of AI products in practice. This work explores two underreported sources of noise that negatively affect imaging AI: (a) variation in labeling schema definitions and (b) noise in the labeling process. First, the overlap between the schemas of two publicly available datasets and a third-party vendor are compared, showing there is low agreement ( 90%). Among low agreement classes (pneumonia, consolidation), the labels assigned as “ground truth” were unreliable, suggesting that the result of majority voting is highly dependent on which group of radiologists is assigned to annotation. Noise in labeling schemas and gold label annotations are pervasive in medical imaging classification and affect downstream clinical deployment. Possible solutions (eg, changes to task design, annotation methods, and model training) and their potential to improve trust in clinical AI are discussed. Keywords: Radiology AI, Dataset Creation, Noise in Datasets Supplemental material is available for this article. © RSNA, 2023 See also the commentary by Ursprung and Woitek in this issue.

Artificial intelligence (AI) software in radiology is becoming increasingly prevalent and performance is improving rapidly with new applications for given use cases being developed continuously, oftentimes with development and validation occurring in parallel. Several guidelines have provided reporting standards for publications of AI-based research in medicine and radiology. Yet, there is an unmet need for recommendations on the assessment of AI software before adoption and after commercialization. As the radiology AI ecosystem continues to grow and mature, a formalization of system assessment and evaluation is paramount to ensure patient safety, relevance and support to clinical workflows, and optimal allocation of limited AI development and validation resources before broader implementation into clinical practice. To fulfil these needs, we provide a glossary for AI software types, use cases and roles within the clinical workflow; list healthcare needs, key performance indicators and required information about software prior to assessment; and lay out examples of software performance metrics per software category. This conceptual framework is intended to streamline communication with the AI software industry and provide healthcare decision makers and radiologists with tools to assess the potential use of these software. The proposed software evaluation framework lays the foundation for a radiologist-led prospective validation network of radiology AI software. Learning Points: The rapid expansion of AI applications in radiology requires standardization of AI software specification, classification, and evaluation. The Canadian Association of Radiologists' AI Tech & Apps Working Group Proposes an AI Specification document format and supports the implementation of a clinical expert evaluation process for Radiology AI software.

Purpose: To observe interactions of practicing radiologists with a chest x-ray AI tool and evaluate its usability and impact on workflow efficiency. Methods: Using a simulated clinical workflow and remote multi-monitor screensharing, we prospectively assessed the interactions of 10 staff radiologists (5–33 years of experience) with a PACS-embedded, regulatory-approved chest x-ray AI tool. Qualitatively, we collected feedback using a think-aloud method and post-testing semi-structured interview; transcript themes were categorized by: (1) AI tool features, (2) deployment considerations, and (3) broad human-AI interactions. Quantitatively, we used time-stamped video recordings to compare reporting and decision-making efficiency with and without AI assistance. Results: For AI tool features, radiologists appreciated the simple binary classification (normal vs abnormal) and found the heatmap essential to understand what the AI considered abnormal; users were uncertain of how to interpret confidence values. Regarding deployment considerations, radiologists thought the tool would be especially helpful for identifying subtle diagnoses; opinions were mixed on whether the tool impacted perceived efficiency, accuracy, and confidence. Considering general human-AI interactions, radiologists shared concerns about automation bias especially when relying on an automated triage function. Regarding decision-making and workflow efficiency, participants began dictating 5 seconds later (42% increase, P = .02) and took 14 seconds longer to complete cases (33% increase, P = .09) with AI assistance. Conclusions: Radiologist usability testing provided insights into effective AI tool features, deployment considerations, and human-AI interactions that can guide successful AI deployment. Early AI adoption may increase radiologists’ decision-making and total reporting time but improves with experience.

Group theory, specifically the combinatorial group theory of finitely presented groups,has been utilized effectively in cryptology. Several new public key cryptosystems have been developed and this...

The overuse of low value medical tests and treatments drives costs and patient harm. Efforts to address overuse, such as Choosing Wisely campaigns, typically rely on passive implementation strategies- a form of low reliability system change. Embedding guidelines into clinical decision support (CDS) software is a higher leverage approach to provide ordering suggestions through an interface embedded within the clinical workflow. Growth in computing power is increasingly enabling artificial intelligence (AI) to augment such decision making tools. This article offers a roadmap of opportunities for AI-enabled CDS to reduce overuse, which are presented according to a patient’s journey of care.

We introduce three first-order languages with equality Ln, n = 0, 1, 2. We list axioms Tn expressed in Ln and view a group as a model of T0 and a ring as a model of T1. Moreover, we view the class ...

INTRODUCTION: We aimed to develop an explainable machine learning (ML) model to predict side-specific extraprostatic extension (ssEPE) to identify patients who can safely undergo nerve-sparing radical prostatectomy using preoperative clinicopathological variables. METHODS: A retrospective sample of clinicopathological data from 900 prostatic lobes at our institution was used as the training cohort. Primary outcome was the presence of ssEPE. The baseline model for comparison had the highest performance out of current biopsy-derived predictive models for ssEPE. A separate logistic regression (LR) model was built using the same variables as the ML model. All models were externally validated using a testing cohort of 122 lobes from another institution. Models were assessed by area under receiver-operating-characteristic curve (AUROC), precision-recall curve (AUPRC), calibration, and decision curve analysis. Model predictions were explained using SHapley Additive exPlanations. This tool was deployed as a publicly available web application. RESULTS: Incidence of ssEPE in the training and testing cohorts were 30.7 and 41.8%, respectively. The ML model achieved AUROC 0.81 (LR 0.78, baseline 0.74) and AUPRC 0.69 (LR 0.64, baseline 0.59) on the training cohort. On the testing cohort, the ML model achieved AUROC 0.81 (LR 0.76, baseline 0.75) and AUPRC 0.78 (LR 0.75, baseline 0.70). The ML model was explainable, well-calibrated, and achieved the highest net benefit for clinically relevant cutoffs of 10–30%. CONCLUSIONS: We developed a user-friendly application that enables physicians without prior ML experience to assess ssEPE risk and understand factors driving these predictions to aid surgical planning and patient counselling (https://share.streamlit.io/jcckwong/ssepe/main/ssEPE_V2.py).

Seamless sharing between imaging facilities of medical images obtained on the same patient is crucial in providing accurate and efficient care to patients. However, the terminology used to describe semantically similar examinations can vary widely between facilities. Current practice is manual table-based mapping to a standard terminology, which has substantial potential for mislabelled and missing examinations. In this work, we establish several baseline methods for automating the mapping of radiology imaging procedure descriptions to a SNOMED CT based standard terminology. Our best performing baseline, consisting of a bag of words representation and shallow neural network, achieved 96.3% accuracy. In addition, we explore an unsupervised clustering method that explores relevancy matching without the need for an intervening standard. Lastly, we make the procedure name dataset used in this work available to encourage extension of this application.

Non-pharmaceutical interventions (NPIs) have been the primary tool used by governments and organizations to mitigate the spread of the ongoing pandemic of COVID-19. Natural experiments are currently being conducted on the impact of these interventions, but most of these occur at the subnational level - data not available in early global datasets. We describe the rapid development of the first comprehensive, labelled dataset of 1640 NPIs implemented at federal, provincial/territorial and municipal levels in Canada to guide COVID-19 research. For each intervention, we provide: a) information on timing to aid in longitudinal evaluation, b) location to allow for robust spatial analyses, and c) classification based on intervention type and target population, including classification aligned with a previously developed measure of government response stringency. This initial dataset release (v1.0) spans January 1st, and March 31st, 2020; bi-weekly data updates to continue for the duration of the pandemic. This novel dataset enables robust, inter-jurisdictional comparisons of pandemic response, can serve as a model for other jurisdictions and can be linked with other information about case counts, transmission dynamics, health care utilization, mobility data and economic indicators to derive important insights regarding NPI impact.

Purpose The aim of this study was to enhance multispecialty CT and MRI protocol assignment quality and efficiency through development, testing, and proposed workflow design of a natural language processing (NLP)–based machine learning classifier. Methods NLP-based machine learning classification models were developed using order entry input data and radiologist-assigned protocols from more than 18,000 unique CT and MRI examinations obtained during routine clinical use. k-Nearest neighbor, random forest, and deep neural network classification models were evaluated at baseline and after applying class frequency and confidence thresholding techniques. To simulate performance in real-world deployment, the model was evaluated in two operating modes in combination: automation (automated assignment of the top result) and clinical decision support (CDS; top-three protocol suggestion for clinical review). Finally, model-radiologist discordance was subjectively reviewed to guide explainability and safe use. Results Baseline protocol assignment performance achieved weighted precision of 0.757 to 0.824. Simulating real-world deployment using combined thresholding techniques, the optimized deep neural network model assigned 69% of protocols in automation mode with 95% accuracy. In the remaining 31% of cases, the model achieved 92% accuracy in CDS mode. Analysis of discordance with subspecialty radiologist labels revealed both more and less appropriate model predictions. Conclusions A multiclass NLP-based classification algorithm was designed to drive local operational improvement in CT and MR radiology protocol assignment at subspecialist quality. The results demonstrate a simulated workflow deployment enabling automated assignment of protocols in nearly 7 of 10 cases with very few errors combined with top-three CDS for remaining cases supporting a high-quality, efficient radiology workflow.

This article is based on a lecture given at the Marx Memorial Library to mark the 150th anniversary of the publication of the first volume of Karl Marx’s Das Kapital and the 200th anniversary of Ma...

Password security is a crucial component of modern internet security. In this paper, we present a provably secure method for password verification using combinatorial group theory. This method relies on the group randomizer system, a subset of the MAGNUS computer algebra system and corrects most of the present problems with challenge response systems, the most common types of password verification. Theoretical security of the considered method depends on several results in asymptotic group theory. We mention further that this method has applications for many other password situations including container security.

The Tarski theorems, proved by Myasnikov and Kharlampovich and inde-pendently by Sela say that all nonabelian free groups satisfy the same first-order or elementary theory. Kharlampovich and Myasnikov also prove that the elementary theory of free groups is decidable. For a group ring they have proved that the first-order theory (in the language of ring theory) is not decidable and have studied equations over group rings, especially for torsion-free hyperbolic groups. In this note we examine and survey extensions of Tarksi-like results to the collection of group rings and examine relationships between the universal and elementary theories of the corresponding groups and rings and the corresponding universal theory of the formed group ring. To accomplish this we introduce different first-order languages with equality whose model classes are respectively groups, rings and group rings. We prove that if R[G] is elementarily equivalent to S[H] then simultaneously the group G is elementarily equivalent to the group H and the ring R is elementarily equivalent to the ring S with respect to the appropriate languages. Further if G is universally equivalent to a nonabelian free group F and R is universally equivalent to the integers Z then R[G] is universally equivalent to Z[F] again with respect to an ap-propriate language.

UNSTRUCTURED Choosing Wisely campaigns have spread to more than 20 countries worldwide. Choosing Wisely is a clinician-led effort to produce clinician recommendations and patient resources around overused tests, procedures and medications. The digitization of medical records and workflows and growth in computing power has enabled artificial intelligence (AI) to be applied to augment health care decision-making. Overuse is an important problem for AI-enabled technologies to address. This commentary offers a roadmap of opportunities for AI-augmented efforts to reduce overuse which are presented according to a patient’s journey of care, beginning from when patients experience symptoms through clinical decision making and subsequent evaluation efforts. This roadmap can be used to guide the cross-discipline development and implementation of AI-enabled tools to reduce overuse and drive value.

In 1985, at Groups St Andrews, Gilbert Baumslag gave a short course on one-relator groups which provided a look at the subject up to that point. In this paper we partially update the massive amount of work done over the past three decades. For the most part we concentrate on areas and results to which the authors have made contributions. We look at the important connections with surface groups and elementary theory, and describe the surface group conjecture and the Gromov conjecture on surface subgroups. We look at the solution by Wise of Baumslag’s residual finiteness conjecture and discuss a new Baumslag conjecture on virtually free-by-cyclic groups. We examine various amalgam decompositions of one-relator groups and the Baumslag-Shalen conjectures. We then look at a series of open problems in one-relator group theory and their status. Finally we introduce a concept called plainarity based on the Magnus breakdown of a one-relator group which might provide a systematic approach to the solution of problems in one-relator groups.

The 1600's ushered in our modern world, but not in the way most people learn in school. There was a revolution; started by Kepler, continued by Galileo, Descartes and Fermat and culminating in Newton and Leibniz. This revolution allowed for the development of modern mathematics which in turn led to modern science and engineering to advance. Hence, the technological revolution occurred which has shaped our present-day existence much more than anything else. In this article we examine these developments during the amazing seventeenth century. We keep an eye on the fact that for whatever reason human beings for the most part seem not to do hard engineering until the hard science is developed and not to do the hard science until the correct mathematics has been discovered.

A theorem of E.C. Turner states that if F is a finitely generated free group, then the test words are precisely the elements not contained in any proper retract. In this paper, we examine some ideas in model theory and logic related to Turner’s characterization of test words and introduce Turner groups, a class of groups containing all finite groups and all stably hyperbolic groups satisfying this characterization. We show that Turner’s theorem is not first-order expressible. However, we prove that every finitely generated elementary free group is a Turner group.

In [5, 6] the relationships between test words, generic elements, almost primitivity and tame almost primitivity were examined in free groups. In this paper we extend the concepts and connections to general free products and in particular to free products of cyclic groups.

A group is G commutative transitive or CT if commuting is transitive on nontrivial elements. A group G is CSA or conjugately separated abelian if maximal abelian subgroups are malnormal. These concepts have played a prominent role in the studies of fully residually free groups, limit groups and discriminating groups. They also play a role in the solution to the Tarski problems. CSA always implies CT however the class of CSA groups is a proper subclass of the class of CT groups. For limit groups and finitely generated elementary free groups they are equivalent. In this paper we examine the relationship between the two concepts. In particular, we show that a finite CSA group must be abelian. If G is CT, then we prove that G is not CSA if and only if G contains a nonabelian subgroup G 0 ${G_{0}}$ which contains a nontrivial abelian subgroup H that is normal in G 0 ${G_{0}}$ . For K a field the group PSL ⁢ ( 2 , K ) ${\mathrm{PSL}(2,K)}$ is never CSA but is CT if char ⁡ ( K ) = 2 ${\operatorname{char}(K)=2}$ and for fields K of characteristic 0 where -1 is not a sum of two squares in K. For characteristic p, for an odd prime p, PSL ⁢ ( 2 , K ) ${\mathrm{PSL}(2,K)}$ is never CT. Infinite CT groups G with a composition series and having no nontrivial normal abelian subgroup must be monolithic with monolith a simple nonabelian CT group. Further if a group G is monolithic with monolith N isomorphic to PSL ⁢ ( 2 , K ) ${\mathrm{PSL}(2,K)}$ for a field K of characteristic 2 and G is CT, then G ≅ N ${G\cong N}$ .