Usability testing of a monitoring and feedback tool to stimulate physical activity

Sanne van der Weegen,1 Renée Verwey,1,2 Huibert J Tange,3 Marieke D Spreeuwenberg,1 Luc P de Witte1,2 1Department of Health Services Research, CAPHRI School for Public Health and Primary Care, Faculty of Health Medicine and Life Sciences, Maastricht University, the Netherlands; 2Research Centre Technology in Care, Zuyd University of Applied Sciences, Heerlen, the Netherlands; 3Department of General Practice, CAPHRI School for Public Health and Primary Care, Faculty of Health Medicine and Life Sciences, Maastricht University, the Netherlands Introduction: A monitoring and feedback tool to stimulate physical activity, consisting of an activity sensor, smartphone application (app), and website for patients and their practice nurses, has been developed: the 'It's LiFe!' tool. In this study the usability of the tool was evaluated by technology experts and end users (people with chronic obstructive pulmonary disease or type 2 diabetes, with ages from 40–70 years), to improve the user interfaces and content of the tool. Patients and methods: The study had four phases: 1) a heuristic evaluation with six technology experts; 2) a usability test in a laboratory by five patients; 3) a pilot in real life wherein 20 patients used the tool for 3 months; and 4) a final lab test by five patients. In both lab tests (phases 2 and 4) qualitative data were collected through a thinking-aloud procedure and video recordings, and quantitative data through questions about task complexity, text comprehensiveness, and readability. In addition, the post-study system usability questionnaire (PSSUQ) was completed for the app and the website. In the pilot test (phase 3), all patients were interviewed three times and the Software Usability Measurement Inventory (SUMI) was completed. Results: After each phase, improvements were made, mainly to the layout and text. The main improvement was a refresh button for active data synchronization between activity sensor, app, and server, implemented after connectivity problems in the pilot test. The mean score on the PSSUQ for the website improved from 5.6 (standard deviation [SD] 1.3) to 6.5 (SD 0.5), and for the app from 5.4 (SD 1.5) to 6.2 (SD 1.1). Satisfaction in the pilot was not very high according to the SUMI. Discussion: The use of laboratory versus real-life tests and expert-based versus user-based tests revealed a wide range of usability issues. The usability of the It's LiFe! tool improved considerably during the study. Keywords: accelerometry, chronic obstructive pulmonary disease, diabetes mellitus type 2, heuristic evaluation, telemonitoring, thinking aloud