Effect of improving the realism of simulated clinical judgement tasks on nurses’ overconfidence and underconfidence: Evidence from a comparative confidence calibration analysis

Background Apparent overconfidence and underconfidence in clinicians making clinical judgements could be a feature of evaluative research designs that fail to accurately represent clinical environments. Objectives To test the effect of improved realism of clinical judgement tasks on confidence calibration performance of nurses and student nurses. Design A comparative confidence calibration analysis. Settings The study was conducted in a large university of Northern England. Methods Ninety-seven participants rated their confidence – using a scale that ranged from 0 (no confidence) to 100 (totally confident) on dichotomous clinical judgements of critical event risk. The judgements were in response to 25 paper-based and 25 higher fidelity scenarios using a computerised patient simulator and clinical equipment. Scenarios, and judgement criteria of ‘correctness', were generated from real patient cases. Using a series of calibration measures (calibration, resolution and over/underconfidence), participants' confidence was calibrated against the proportion of correct judgements. The calibration measures generated by the paper-based and high fidelity clinical simulation conditions were compared. Results Participants made significantly less accurate clinical judgements of risk in the high fidelity clinical simulations compared to the paper simulations ( P =0.0002). They were significantly less confident in high fidelity clinical simulations than paper simulations ( P =0.03). However, there was no significant difference of over/underconfidence for participants between the two simulated settings ( P =0.06). Participants were no better calibrated in the high fidelity clinical simulations than paper simulations, P =0.85. Likewise, participants had no better ability of discriminating correct judgements from incorrect judgements as measured by the resolution statistic in high fidelity clinical simulations than paper simulations, P =0.76. Conclusions Improving the realism of simulated judgement tasks led to reduced confidence and judgement accuracy in participants but did not alter confidence calibration. These findings suggest that judgemental miscalibration of confidence in nurses may be a systematic cognitive bias and that simply making scenarios more realistic may not be a sufficient condition for correction.