Search

Radiology reports are a valuable source of textual information used to improve clinical care and support research. In recent years, deep learning techniques have been shown to be effective in classifying radiology reports. This article investigates the use of deep learning techniques with attention mechanisms to achieve better performance in the classification of radiology reports. We focus on various Natural Language Processing approaches, such as LSTM with Attention, BERT, and GPT-4, evaluated on a chest tomography report dataset regarding neoplastic diseases collected from an Italian hospital. In particular, we compare the results with a previous machine learning system, showing that models based on attention mechanisms can achieve higher performance. The Attention Mechanism allows us to identify the most relevant bits of text used by the model to make its predictions. We show that our model achieves state-of-the-art results on the hierarchical classification of radiology reports. Moreover, we evaluate the performance of GPT-4 on the classification of these reports in a zero-shot setup through prompt engineering, showing interesting results even with a small context and a non-English language. Our findings suggest that deep learning techniques with attention mechanisms may be successful in the classification of radiology reports even in non-English languages for which it is not possible to leverage on large text corpus.

The analysis of the presence of bias, prejudices and unwanted discriminatory behavior in pre-trained neural language models (NLMs), considering the sensitivity of the topic and its public interest, should respect two main criteria: the intuition and the statistical rigor. To the state of the art, there are two main categories of approaches for analyzing bias: those based on the models’ textual output, and those based on the geometric space of the embedded representations calculated by the NLMs. While the first one is intuitive, this kind of analysis is often conducted on simple template sentences, which limit the overall validity of their conclusions in a real-world context. On the contrary, geometric methods are more rigorous but quite more complex to implement and understand for those who are non-experts in Natural Language Processing (NLP). In this paper, we propose a unique method for analyzing bias in pre-trained language models that combines these two aspects. Through a simple classification task, we verify whether the information contained in the embedded representation of words that describes a protected property (such as the religion) can be used to identify a stereotyped property (such as the criminal behavior), requiring only a minimal supervised dataset. We experimentally verify our approach, finding that four widespread Transformer-based models are affected by prejudices of gender, nationality, and religion.

AI and Machine Learning (ML) offer powerful tools to support clinical decision making in emergency situations such as the COVID-19 pandemic. In this context, the application of ML requires to design predictive systems that have adequate accuracy and can effectively deal with issues concerning data quality, sensitive errors, uncertainty, and interpretability of the predictions. We present a methodology that deals with all these problems and a concrete study of its application to estimate the prognosis of hospitalised patients with COVID-19. In particular, we address the task of predicting the outcome (alive or deceased) of a patient at different times of her/his hospitalisation minimising false negatives (wrong survival predictions). The proposed methodology builds different optimised ML models to select those that perform the best to recognise, at different times of hospitalisation, patients who will have an unfavourable prognosis (decease). These models exploit a new algorithm, presented in the paper, that identifies an uncertainty threshold to rule out uncertain predictions with the purpose of making a ML model both more performing and more reliable. Moreover, we propose a general method for automatically extracting multi-variable prognostic rules from the available data. Such rules can provide possible new useful knowledge on the considered disease. We also show how they can be used effectively to explain the predictions made by the ML models. All proposed methods and techniques are experimentally evaluated in the context of our application task.