Search

Your search keyword '"Campagner, Andrea"' showing total 293 results
Start Over Author "Campagner, Andrea"
293 results on '"Campagner, Andrea"'

2. Dissimilar Similarities: Comparing Human and Statistical Similarity Evaluation in Medical AI

Author

Cabitza, Federico, Famiglini, Lorenzo, Campagner, Andrea, Sconfienza, Luca Maria, Fusco, Stefano, Caccavella, Valerio, Gallazzi, Enrico, Goos, Gerhard, Series Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Torra, Vicenç, editor, Narukawa, Yasuo, editor, and Kikuchi, Hiroaki, editor

Published
2024
Full Text
View/download PDF

5. Painting the black box white: experimental findings from applying XAI to an ECG reading setting

Author

Cabitza, Federico, Cameli, Matteo, Campagner, Andrea, Natali, Chiara, and Ronzio, Luca

Subjects
Computer Science - Artificial Intelligence, I.2.1, H.5.2
Abstract

The shift from symbolic AI systems to black-box, sub-symbolic, and statistical ones has motivated a rapid increase in the interest toward explainable AI (XAI), i.e. approaches to make black-box AI systems explainable to human decision makers with the aim of making these systems more acceptable and more usable tools and supports. However, we make the point that, rather than always making black boxes transparent, these approaches are at risk of \emph{painting the black boxes white}, thus failing to provide a level of transparency that would increase the system's usability and comprehensibility; or, even, at risk of generating new errors, in what we termed the \emph{white-box paradox}. To address these usability-related issues, in this work we focus on the cognitive dimension of users' perception of explanations and XAI systems. To this aim, we designed and conducted a questionnaire-based experiment by which we involved 44 cardiology residents and specialists in an AI-supported ECG reading task. In doing so, we investigated different research questions concerning the relationship between users' characteristics (e.g. expertise) and their perception of AI and XAI systems, including their trust, the perceived explanations' quality and their tendency to defer the decision process to automation (i.e. technology dominance), as well as the mutual relationships among these different dimensions. Our findings provide a contribution to the evaluation of AI-based support systems from a Human-AI interaction-oriented perspective and lay the ground for further investigation of XAI and its effects on decision making and user experience., Comment: 15 pages, 7 figures

Published
2022

6. Everything is Varied: The Surprising Impact of Individual Variation on ML Robustness in Medicine

Author

Campagner, Andrea, Famiglini, Lorenzo, Carobene, Anna, and Cabitza, Federico

Subjects
Computer Science - Machine Learning
Abstract

In medical settings, Individual Variation (IV) refers to variation that is due not to population differences or errors, but rather to within-subject variation, that is the intrinsic and characteristic patterns of variation pertaining to a given instance or the measurement process. While taking into account IV has been deemed critical for proper analysis of medical data, this source of uncertainty and its impact on robustness have so far been neglected in Machine Learning (ML). To fill this gap, we look at how IV affects ML performance and generalization and how its impact can be mitigated. Specifically, we provide a methodological contribution to formalize the problem of IV in the statistical learning framework and, through an experiment based on one of the largest real-world laboratory medicine datasets for the problem of COVID-19 diagnosis, we show that: 1) common state-of-the-art ML models are severely impacted by the presence of IV in data; and 2) advanced learning strategies, based on data augmentation and data imprecisiation, and proper study designs can be effective at improving robustness to IV. Our findings demonstrate the critical relevance of correctly accounting for IV to enable safe deployment of ML in clinical settings.

Published
2022

7. A Distributional Approach for Soft Clustering Comparison and Evaluation

Author

Campagner, Andrea, Ciucci, Davide, and Denœux, Thierry

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence
Abstract

The development of external evaluation criteria for soft clustering (SC) has received limited attention: existing methods do not provide a general approach to extend comparison measures to SC, and are unable to account for the uncertainty represented in the results of SC algorithms. In this article, we propose a general method to address these limitations, grounding on a novel interpretation of SC as distributions over hard clusterings, which we call \emph{distributional measures}. We provide an in-depth study of complexity- and metric-theoretic properties of the proposed approach, and we describe approximation techniques that can make the calculations tractable. Finally, we illustrate our approach through a simple but illustrative experiment., Comment: This is the extended version of article "A Distributional Approach for Soft Clustering Comparison and Evaluation", accepted at BELIEF 2022 (http://hebergement.universite-paris-saclay.fr/belief2022/). Please cite the proceedings version of the article

Published
2022

8. Unity Is Intelligence: A Collective Intelligence Experiment on ECG Reading to Improve Diagnostic Performance in Cardiology

Author

Ronzio, Luca, Campagner, Andrea, Cabitza, Federico, and Gensini, Gian Franco

Abstract

Medical errors have a huge impact on clinical practice in terms of economic and human costs. As a result, technology-based solutions, such as those grounded in artificial intelligence (AI) or collective intelligence (CI), have attracted increasing interest as a means of reducing error rates and their impacts. Previous studies have shown that a combination of individual opinions based on rules, weighting mechanisms, or other CI solutions could improve diagnostic accuracy with respect to individual doctors. We conducted a study to investigate the potential of this approach in cardiology and, more precisely, in electrocardiogram (ECG) reading. To achieve this aim, we designed and conducted an experiment involving medical students, recent graduates, and residents, who were asked to annotate a collection of 10 ECGs of various complexity and difficulty. For each ECG, we considered groups of increasing size (from three to 30 members) and applied three different CI protocols. In all cases, the results showed a statistically significant improvement (ranging from 9% to 88%) in terms of diagnostic accuracy when compared to the performance of individual readers; this difference held for not only large groups, but also smaller ones. In light of these results, we conclude that CI approaches can support the tasks mentioned above, and possibly other similar ones as well. We discuss the implications of applying CI solutions to clinical settings, such as cases of augmented 'second opinions' and decision-making.

Published
2021

9. Aggregation Operators on Shadowed Sets Deriving from Conditional Events and Consensus Operators

Author

Boffa, Stefania, Campagner, Andrea, Ciucci, Davide, Yao, Yiyu, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Campagner, Andrea, editor, Urs Lenz, Oliver, editor, Xia, Shuyin, editor, Ślęzak, Dominik, editor, Wąs, Jarosław, editor, and Yao, JingTao, editor

Published
2023
Full Text
View/download PDF

10. The Tower of Babel in Explainable Artificial Intelligence (XAI)

Author

Schneeberger, David, Röttger, Richard, Cabitza, Federico, Campagner, Andrea, Plass, Markus, Müller, Heimo, Holzinger, Andreas, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Kieseberg, Peter, editor, Cabitza, Federico, editor, Campagner, Andrea, editor, Tjoa, A Min, editor, and Weippl, Edgar, editor

Published
2023
Full Text
View/download PDF

11. Controllable AI - An Alternative to Trustworthiness in Complex AI Systems?

Author

Kieseberg, Peter, Weippl, Edgar, Tjoa, A. Min, Cabitza, Federico, Campagner, Andrea, Holzinger, Andreas, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Kieseberg, Peter, editor, Cabitza, Federico, editor, Campagner, Andrea, editor, Tjoa, A Min, editor, and Weippl, Edgar, editor

Published
2023
Full Text
View/download PDF

12. Let Me Think! Investigating the Effect of Explanations Feeding Doubts About the AI Advice

Author

Cabitza, Federico, Campagner, Andrea, Famiglini, Lorenzo, Natali, Chiara, Caccavella, Valerio, Gallazzi, Enrico, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Kieseberg, Peter, editor, Cabitza, Federico, editor, Campagner, Andrea, editor, Tjoa, A Min, editor, and Weippl, Edgar, editor

Published
2023
Full Text
View/download PDF

14. Toward a Perspectivist Turn in Ground Truthing for Predictive Computing

Author

Basile, Valerio, Cabitza, Federico, Campagner, Andrea, and Fell, Michael

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence
Abstract

Most Artificial Intelligence applications are based on supervised machine learning (ML), which ultimately grounds on manually annotated data. The annotation process is often performed in terms of a majority vote and this has been proved to be often problematic, as highlighted by recent studies on the evaluation of ML models. In this article we describe and advocate for a different paradigm, which we call data perspectivism, which moves away from traditional gold standard datasets, towards the adoption of methods that integrate the opinions and perspectives of the human subjects involved in the knowledge representation step of ML processes. Drawing on previous works which inspired our proposal we describe the potential of our proposal for not only the more subjective tasks (e.g. those related to human language) but also to tasks commonly understood as objective (e.g. medical decision making), and present the main advantages of adopting a perspectivist stance in ML, as well as possible disadvantages, and various ways in which such a stance can be implemented in practice. Finally, we share a set of recommendations and outline a research agenda to advance the perspectivist stance in ML., Comment: If you wish to cite this work, consider citing the AAAI 2023 proceedings version (https://doi.org/10.1609/aaai.v37i6.25840) and citing it in this way: Cabitza, F., Campagner, A., & Basile, V. (2023). Toward a Perspectivist Turn in Ground Truthing for Predictive Computing. Proceedings of the AAAI Conference on Artificial Intelligence, 37(6), 6860-6868. https://doi.org/10.1609/aaai.v37i6.25840

Published
2021
Full Text
View/download PDF

19. The Impact of Gender and Personality in Human-AI Teaming: The Case of Collaborative Question Answering

Author

Milella, Frida, Natali, Chiara, Scantamburlo, Teresa, Campagner, Andrea, Cabitza, Federico, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Abdelnour Nocera, José, editor, Kristín Lárusdóttir, Marta, editor, Petrie, Helen, editor, Piccinno, Antonio, editor, and Winckler, Marco, editor

Published
2023
Full Text
View/download PDF

23. Who wants accurate models? Arguing for a different metrics to take classification models seriously

Author

Cabitza, Federico and Campagner, Andrea

Subjects
Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

With the increasing availability of AI-based decision support, there is an increasing need for their certification by both AI manufacturers and notified bodies, as well as the pragmatic (real-world) validation of these systems. Therefore, there is the need for meaningful and informative ways to assess the performance of AI systems in clinical practice. Common metrics (like accuracy scores and areas under the ROC curve) have known problems and they do not take into account important information about the preferences of clinicians and the needs of their specialist practice, like the likelihood and impact of errors and the complexity of cases. In this paper, we present a new accuracy measure, the H-accuracy (Ha), which we claim is more informative in the medical domain (and others of similar needs) for the elements it encompasses. We also provide proof that the H-accuracy is a generalization of the balanced accuracy and establish a relation between the H-accuracy and the Net Benefit. Finally, we illustrate an experimentation in two user studies to show the descriptive power of the Ha score and how complementary and differently informative measures can be derived from its formulation (a Python script to compute Ha is also made available)., Comment: https://github.com/AndreaCampagner/uncertainpy

Published
2019

28. Scikit-Weak: A Python Library for Weakly Supervised Machine Learning

Author

Campagner, Andrea, Lienen, Julian, Hüllermeier, Eyke, Ciucci, Davide, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Yao, JingTao, editor, Fujita, Hamido, editor, Yue, Xiaodong, editor, Miao, Duoqian, editor, Grzymala-Busse, Jerzy, editor, and Li, Fanzhang, editor

Published
2022
Full Text
View/download PDF

29. Orthopartitions in Knowledge Representation and Machine Learning

Author

Ciucci, Davide, Boffa, Stefania, Campagner, Andrea, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Yao, JingTao, editor, Fujita, Hamido, editor, Yue, Xiaodong, editor, Miao, Duoqian, editor, Grzymala-Busse, Jerzy, editor, and Li, Fanzhang, editor

Published
2022
Full Text
View/download PDF

30. Re-calibrating Machine Learning Models Using Confidence Interval Bounds

Author

Campagner, Andrea, Famiglini, Lorenzo, Cabitza, Federico, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Torra, Vicenç, editor, and Narukawa, Yasuo, editor

Published
2022
Full Text
View/download PDF

31. Color Shadows (Part I): Exploratory Usability Evaluation of Activation Maps in Radiological Machine Learning

Author

Cabitza, Federico, Campagner, Andrea, Famiglini, Lorenzo, Gallazzi, Enrico, La Maida, Giovanni Andrea, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Kieseberg, Peter, editor, Tjoa, A Min, editor, and Weippl, Edgar, editor

Published
2022
Full Text
View/download PDF

32. Global Interpretable Calibration Index, a New Metric to Estimate Machine Learning Models’ Calibration

Author

Cabitza, Federico, Campagner, Andrea, Famiglini, Lorenzo, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Kieseberg, Peter, editor, Tjoa, A Min, editor, and Weippl, Edgar, editor

Published
2022
Full Text
View/download PDF

33. Rough-set Based Genetic Algorithms for Weakly Supervised Feature Selection

Author

Campagner, Andrea, Ciucci, Davide, Filipe, Joaquim, Editorial Board Member, Ghosh, Ashish, Editorial Board Member, Prates, Raquel Oliveira, Editorial Board Member, Zhou, Lizhu, Editorial Board Member, Ciucci, Davide, editor, Couso, Inés, editor, Medina, Jesús, editor, Ślęzak, Dominik, editor, Petturiti, Davide, editor, Bouchon-Meunier, Bernadette, editor, and Yager, Ronald R., editor

Published
2022
Full Text
View/download PDF

42. Feature Selection and Disambiguation in Learning from Fuzzy Labels Using Rough Sets

Author

Campagner, Andrea, Ciucci, Davide, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Ramanna, Sheela, editor, Cornelis, Chris, editor, and Ciucci, Davide, editor

Published
2021
Full Text
View/download PDF

43. Weighted Utility: A Utility Metric Based on the Case-Wise Raters’ Perceptions

Author

Campagner, Andrea, Conte, Enrico, Cabitza, Federico, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Kieseberg, Peter, editor, Tjoa, A Min, editor, and Weippl, Edgar, editor

Published
2021
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results