1. Questioning AI: Promoting Decision-Making Autonomy Through Reflection
- Author
-
Fischer, Simon WS
- Subjects
Computer Science - Human-Computer Interaction - Abstract
Decision-making is increasingly supported by machine recommendations. In healthcare, for example, a clinical decision support system is used by the physician to find a treatment option for a patient. In doing so, people can rely too much on these systems, which impairs their own reasoning process. The European AI Act addresses the risk of over-reliance and postulates in Article 14 on human oversight that people should be able "to remain aware of the possible tendency of automatically relying or over-relying on the output". Similarly, the EU High-Level Expert Group identifies human agency and oversight as the first of seven key requirements for trustworthy AI. The following position paper proposes a conceptual approach to generate machine questions about the decision at hand, in order to promote decision-making autonomy. This engagement in turn allows for oversight of recommender systems. The systematic and interdisciplinary investigation (e.g., machine learning, user experience design, psychology, philosophy of technology) of human-machine interaction in relation to decision-making provides insights to questions like: how to increase human oversight and calibrate over- and under-reliance on machine recommendations; how to increase decision-making autonomy and remain aware of other possibilities beyond automated suggestions that repeat the status-quo?, Comment: Accepted at the 7th FAccTRec Workshop on Responsible Recommendation at ACM RecSys 2024
- Published
- 2024