Romaric Marcilly, Wu-Yi Zheng, Paul Quindroit, Sylvia Pelayo, Sarah Berdot, Bruno Charpiat, Jennifer Corny, Sylvain Drouot, Pauline Frery, Géraldine Leguelinel-Blache, Lisa Mondet, Arnaud Potier, Laurine Robert, Laurie Ferret, Melissa Baysari, CHU Lille, Evaluation des technologies de santé et des pratiques médicales - ULR 2694 (METRICS), Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille), Centre d'Investigation Clinique - Innovation Technologique de Lille - CIC 1403 - CIC 9301 (CIC Lille), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Lille-Centre Hospitalier Régional Universitaire [Lille] (CHRU Lille), Blackdog institute, Hôpital Européen Georges Pompidou [APHP] (HEGP), Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP)-Hôpitaux Universitaires Paris Ouest - Hôpitaux Universitaires Île de France Ouest (HUPO), Health data- and model- driven Knowledge Acquisition (HeKA), Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre de Recherche des Cordeliers (CRC (UMR_S_1138 / U1138)), École Pratique des Hautes Études (EPHE), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Sorbonne Université (SU)-Université Paris Cité (UPCité)-École Pratique des Hautes Études (EPHE), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Sorbonne Université (SU)-Université Paris Cité (UPCité), Centre de Recherche des Cordeliers (CRC (UMR_S_1138 / U1138)), Hôpital de la Croix-Rousse [CHU - HCL], Hospices Civils de Lyon (HCL), Centre hospitalier Saint-Joseph [Paris], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP), Hôpital Bicêtre, CHR de Metz-Thionville, Institut Desbrest de santé publique (IDESP), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Montpellier (UM), Hôpital Universitaire Carémeau [Nîmes] (CHU Nîmes), Centre Hospitalier Universitaire de Nîmes (CHU Nîmes), CHU Amiens-Picardie, Centre Hospitalier Régional Universitaire de Nancy (CHRU Nancy), Centre hospitalier de Lunéville (GHEMM ), Centre hospitalier [Valenciennes, Nord], and The University of Sydney
International audience; Objective: Two tools are currently available in the literature to evaluate the usability of medication alert systems, the instrument for evaluating human factors principles in medication-related decision support alerts (I-MeDeSA) and the tool for evaluating medication alerting systems (TEMAS). This study aimed to compare their convergent validity, perceived usability, usefulness, strengths, and weaknesses, as well as users' preferences.Method: To evaluate convergent validity, two experts mapped TEMAS' items against I-MeDeSA's items with respect to the usability dimensions they target. To assess perceived usability, usefulness, strengths, and weaknesses of both tools, staff with expertise in their medication alerting system were asked to use French versions of the TEMAS and I-MeDeSA. After the use of each tool, participants were asked to complete the System Usability Scale (SUS) and answer questions about the understandability and usefulness of each tool. Finally, participants were asked to name their preferred tool. Numeric scores were statistically compared. Free-text responses were analyzed using an inductive approach.Results: Forty-five participants from 10 hospitals took part in the study. In terms of convergent validity, I-MeDeSA focuses more on the usability of the graphical user interface while TEMAS considers a wider range of usability principles. Both tools have a fair level of perceived usability (I-MeDeSA' SUS score = 61.85 and TEMAS' SUS score = 62.87), but results highlight that revisions are necessary to both tools to improve their usability. Participants found TEMAS more useful than I-MeDeSA (t = -3.63, p =.005) and had a clear preference for TEMAS to identify problems in formative evaluation (39 of 45; 0.867, p