Search

Abstract We studied clinical AI-supported decision-making as an example of a high-stakes setting in which explainable AI (XAI) has been proposed as useful (by theoretically providing physicians with context for the AI suggestion and thereby helping them to reject unsafe AI recommendations). Here, we used objective neurobehavioural measures (eye-tracking) to see how physicians respond to XAI with N = 19 ICU physicians in a hospital’s clinical simulation suite. Prescription decisions were made both pre- and post-reveal of either a safe or unsafe AI recommendation and four different types of simultaneously presented XAI. We used overt visual attention as a marker for where physician mental attention was directed during the simulations. Unsafe AI recommendations attracted significantly greater attention than safe AI recommendations. However, there was no appreciably higher level of attention placed onto any of the four types of explanation during unsafe AI scenarios (i.e. XAI did not appear to ‘rescue’ decision-makers). Furthermore, self-reported usefulness of explanations by physicians did not correlate with the level of attention they devoted to the explanations reinforcing the notion that using self-reports alone to evaluate XAI tools misses key aspects of the interaction behaviour between human and machine.

Abstract The influence of AI recommendations on physician behaviour remains poorly characterised. We assess how clinicians’ decisions may be influenced by additional information more broadly, and how this influence can be modified by either the source of the information (human peers or AI) and the presence or absence of an AI explanation (XAI, here using simple feature importance). We used a modified between-subjects design where intensive care doctors (N = 86) were presented on a computer for each of 16 trials with a patient case and prompted to prescribe continuous values for two drugs. We used a multi-factorial experimental design with four arms, where each clinician experienced all four arms on different subsets of our 24 patients. The four arms were (i) baseline (control), (ii) peer human clinician scenario showing what doses had been prescribed by other doctors, (iii) AI suggestion and (iv) XAI suggestion. We found that additional information (peer, AI or XAI) had a strong influence on prescriptions (significantly for AI, not so for peers) but simple XAI did not have higher influence than AI alone. There was no correlation between attitudes to AI or clinical experience on the AI-supported decisions and nor was there correlation between what doctors self-reported about how useful they found the XAI and whether the XAI actually influenced their prescriptions. Our findings suggest that the marginal impact of simple XAI was low in this setting and we also cast doubt on the utility of self-reports as a valid metric for assessing XAI in clinical experts.

Abstract Long duration spaceflights to the Moon or Mars are at risk for emergency medical events. Managing a hypoxemic distress and performing an advanced airway procedure such as oro-tracheal intubation may be complicated under weightlessness due to ergonomic constraints. An emergency free-floating intubation would be dangerous because of high failure rates due to stabilization issues that prohibits its implementation in a space environment. Nevertheless, we hypothesized that two configurations could lead to a high first-pass success score for intubation performed by a free-floating operator. In a non-randomized, controlled, cross-over simulation study during a parabolic flight campaign, we evaluated and compared the intubation performance of free-floating trained operators, using either a conventional direct laryngoscope in an ice-pick position or an indirect laryngoscopy with a video-laryngoscope in a classic position at the head of a high-fidelity simulation manikin, in weightlessness and in normogravity. Neither of the two tested conditions reached the minimal terrestrial ILCOR recommendations (95% first-pass success) and therefore could not be recommended for general implementation under weightlessness conditions. Free-floating video laryngoscopy at the head of the manikin had a significant better success score than conventional direct laryngoscopy in an ice-pick position. Our results, combined with the preexisting literature, emphasis the difficulties of performing oro-tracheal intubation, even for experts using modern airway devices, under postural instability in weightlessness. ClinicalTrials registration number NCT05303948.