Search

Deep neural networks have been widely adopted in numerous domains due to their high performance and accessibility to developers and application-specific end-users. Fundamental to image-based applications is the development of Convolutional Neural Networks (CNNs), which possess the ability to automatically extract features from data. However, comprehending these complex models and their learned representations, which typically comprise millions of parameters and numerous layers, remains a challenge for both developers and end-users. This challenge arises due to the absence of interpretable and transparent tools to make sense of black-box models. There exists a growing body of Explainable Artificial Intelligence (XAI) literature, including a collection of methods denoted Class Activation Maps (CAMs), that seek to demystify what representations the model learns from the data, how it informs a given prediction, and why it, at times, performs poorly in certain tasks. We propose a novel XAI visualization method denoted CAManim that seeks to simultaneously broaden and focus end-user understanding of CNN predictions by animating the CAM-based network activation maps through all layers, effectively depicting from end-to-end how a model progressively arrives at the final layer activation. Herein, we demonstrate that CAManim works with any CAM-based method and various CNN architectures. Beyond qualitative model assessments, we additionally propose a novel quantitative assessment that expands upon the Remove and Debias (ROAD) metric, pairing the qualitative end-to-end network visual explanations assessment with our novel quantitative "yellow brick ROAD" assessment (ybROAD). This builds upon prior research to address the increasing demand for interpretable, robust, and transparent model assessment methodology, ultimately improving an end-user's trust in a given model's predictions.

The need for clear, trustworthy explanations of deep learning model predictions is essential for high-criticality fields, such as medicine and biometric identification. Class Activation Maps (CAMs) are an increasingly popular category of visual explanation methods for Convolutional Neural Networks (CNNs). However, the performance of individual CAMs depends largely on experimental parameters such as the selected image, target class, and model. Here, we propose MetaCAM, an ensemble-based method for combining multiple existing CAM methods based on the consensus of the top-k% most highly activated pixels across component CAMs. We perform experiments to quantifiably determine the optimal combination of 11 CAMs for a given MetaCAM experiment. A new method denoted Cumulative Residual Effect (CRE) is proposed to summarize large-scale ensemble-based experiments. We also present adaptive thresholding and demonstrate how it can be applied to individual CAMs to improve their performance, measured using pixel perturbation method Remove and Debias (ROAD). Lastly, we show that MetaCAM outperforms existing CAMs and refines the most salient regions of images used for model predictions. In a specific example, MetaCAM improved ROAD performance to 0.393 compared to 11 individual CAMs with ranges from -0.101-0.172, demonstrating the importance of combining CAMs through an ensembling method and adaptive thresholding., Comment: 9 pages

Optimal performance is critical for decision-making tasks from medicine to autonomous driving, however common performance measures may be too general or too specific. For binary classifiers, diagnostic tests or prognosis at a timepoint, measures such as the area under the receiver operating characteristic curve, or the area under the precision recall curve, are too general because they include unrealistic decision thresholds. On the other hand, measures such as accuracy, sensitivity or the F1 score are measures at a single threshold that reflect an individual single probability or predicted risk, rather than a range of individuals or risk. We propose a method in between, deep ROC analysis, that examines groups of probabilities or predicted risks for more insightful analysis. We translate esoteric measures into familiar terms: AUC and the normalized concordant partial AUC are balanced average accuracy (a new finding); the normalized partial AUC is average sensitivity; and the normalized horizontal partial AUC is average specificity. Along with post-test measures, we provide a method that can improve model selection in some cases and provide interpretation and assurance for patients in each risk group. We demonstrate deep ROC analysis in two case studies and provide a toolkit in Python., Comment: 14 pages, 6 Figures, submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), currently under review

Introduction: Adverse drug events (ADEs) are common in older people and contribute significantly to emergency department (ED) visits, unplanned hospitalisations, healthcare costs, morbidity and mortality. Many ADEs are avoidable if attention is directed towards identifying and preventing inappropriate drug use and undesirable drug combinations. Tools exist to identify potentially inappropriate prescribing (PIP) in clinical settings, but they are underused. Applying PIP assessment tools to population-wide health administrative data could provide an opportunity to assess the impact of PIP on individual patients as well as on the healthcare system. This would open new possibilities for interventions to monitor and optimise medication management on a broader, population-level scale. Methods and analysis: The aim of this study is to describe the occurrence of PIP in Ontario's older population (aged 65 years and older), and to assess the health outcomes and health system costs associated with PIP-more specifically, the association between PIP and the occurrence of ED visits, hospitalisations and death, and their related costs. This will be done within the framework of a population-based retrospective cohort study using Ontario's large health administrative and population databases. Eligible patients aged 66 years and older who were issued at least 1 prescription between 1 April 2003 and 31 March 2014 (approximately 2 million patients) will be included. Ethics and dissemination: Ethical approval was obtained from the Ottawa Health Services Network Ethical Review Board and from the Bruyere Research Institute Ethics Review Board. Dissemination will occur via publication, presentation at national and international conferences, and ongoing exchanges with regional, provincial and national stakeholders, including the Ontario Drug Policy Research Network and the Ontario Ministry of Health and Long-Term Care.

Introduction: Adverse drug events (ADEs) are common in older people and contribute significantly to emergency department (ED) visits, unplanned hospitalisations, healthcare costs, morbidity and mortality. Many ADEs are avoidable if attention is directed towards identifying and preventing inappropriate drug use and undesirable drug combinations. Tools exist to identify potentially inappropriate prescribing (PIP) in clinical settings, but they are underused. Applying PIP assessment tools to population-wide health administrative data could provide an opportunity to assess the impact of PIP on individual patients as well as on the healthcare system. This would open new possibilities for interventions to monitor and optimise medication management on a broader, population-level scale. Methods and analysis: The aim of this study is to describe the occurrence of PIP in Ontario's older population (aged 65 years and older), and to assess the health outcomes and health system costs associated with PIP-more specifically, the association between PIP and the occurrence of ED visits, hospitalisations and death, and their related costs. This will be done within the framework of a population-based retrospective cohort study using Ontario's large health administrative and population databases. Eligible patients aged 66 years and older who were issued at least 1 prescription between 1 April 2003 and 31 March 2014 (approximately 2 million patients) will be included. Ethics and dissemination: Ethical approval was obtained from the Ottawa Health Services Network Ethical Review Board and from the Bruyere Research Institute Ethics Review Board. Dissemination will occur via publication, presentation at national and international conferences, and ongoing exchanges with regional, provincial and national stakeholders, including the Ontario Drug Policy Research Network and the Ontario Ministry of Health and Long-Term Care.

Introduction: Adverse drug events (ADEs) are common in older people and contribute significantly to emergency department (ED) visits, unplanned hospitalisations, healthcare costs, morbidity and mortality. Many ADEs are avoidable if attention is directed towards identifying and preventing inappropriate drug use and undesirable drug combinations. Tools exist to identify potentially inappropriate prescribing (PIP) in clinical settings, but they are underused. Applying PIP assessment tools to population-wide health administrative data could provide an opportunity to assess the impact of PIP on individual patients as well as on the healthcare system. This would open new possibilities for interventions to monitor and optimise medication management on a broader, population-level scale. Methods and analysis: The aim of this study is to describe the occurrence of PIP in Ontario's older population (aged 65 years and older), and to assess the health outcomes and health system costs associated with PIP-more specifically, the association between PIP and the occurrence of ED visits, hospitalisations and death, and their related costs. This will be done within the framework of a population-based retrospective cohort study using Ontario's large health administrative and population databases. Eligible patients aged 66 years and older who were issued at least 1 prescription between 1 April 2003 and 31 March 2014 (approximately 2 million patients) will be included. Ethics and dissemination: Ethical approval was obtained from the Ottawa Health Services Network Ethical Review Board and from the Bruyere Research Institute Ethics Review Board. Dissemination will occur via publication, presentation at national and international conferences, and ongoing exchanges with regional, provincial and national stakeholders, including the Ontario Drug Policy Research Network and the Ontario Ministry of Health and Long-Term Care.

Introduction: Adverse drug events (ADEs) are common in older people and contribute significantly to emergency department (ED) visits, unplanned hospitalisations, healthcare costs, morbidity and mortality. Many ADEs are avoidable if attention is directed towards identifying and preventing inappropriate drug use and undesirable drug combinations. Tools exist to identify potentially inappropriate prescribing (PIP) in clinical settings, but they are underused. Applying PIP assessment tools to population-wide health administrative data could provide an opportunity to assess the impact of PIP on individual patients as well as on the healthcare system. This would open new possibilities for interventions to monitor and optimise medication management on a broader, population-level scale. Methods and analysis: The aim of this study is to describe the occurrence of PIP in Ontario's older population (aged 65 years and older), and to assess the health outcomes and health system costs associated with PIP-more specifically, the association between PIP and the occurrence of ED visits, hospitalisations and death, and their related costs. This will be done within the framework of a population-based retrospective cohort study using Ontario's large health administrative and population databases. Eligible patients aged 66 years and older who were issued at least 1 prescription between 1 April 2003 and 31 March 2014 (approximately 2 million patients) will be included. Ethics and dissemination: Ethical approval was obtained from the Ottawa Health Services Network Ethical Review Board and from the Bruyere Research Institute Ethics Review Board. Dissemination will occur via publication, presentation at national and international conferences, and ongoing exchanges with regional, provincial and national stakeholders, including the Ontario Drug Policy Research Network and the Ontario Ministry of Health and Long-Term Care.

Introduction: Adverse drug events (ADEs) are common in older people and contribute significantly to emergency department (ED) visits, unplanned hospitalisations, healthcare costs, morbidity and mortality. Many ADEs are avoidable if attention is directed towards identifying and preventing inappropriate drug use and undesirable drug combinations. Tools exist to identify potentially inappropriate prescribing (PIP) in clinical settings, but they are underused. Applying PIP assessment tools to population-wide health administrative data could provide an opportunity to assess the impact of PIP on individual patients as well as on the healthcare system. This would open new possibilities for interventions to monitor and optimise medication management on a broader, population-level scale. Methods and analysis: The aim of this study is to describe the occurrence of PIP in Ontario's older population (aged 65 years and older), and to assess the health outcomes and health system costs associated with PIP-more specifically, the association between PIP and the occurrence of ED visits, hospitalisations and death, and their related costs. This will be done within the framework of a population-based retrospective cohort study using Ontario's large health administrative and population databases. Eligible patients aged 66 years and older who were issued at least 1 prescription between 1 April 2003 and 31 March 2014 (approximately 2 million patients) will be included. Ethics and dissemination: Ethical approval was obtained from the Ottawa Health Services Network Ethical Review Board and from the Bruyere Research Institute Ethics Review Board. Dissemination will occur via publication, presentation at national and international conferences, and ongoing exchanges with regional, provincial and national stakeholders, including the Ontario Drug Policy Research Network and the Ontario Ministry of Health and Long-Term Care.