Search

Background: Climate change and increased awareness of planetary health have made reducing ecological footprints a priority for healthcare organizations. However, improving healthcare's environmental impact remains difficult. Numerous researchers argue these difficulties are caused by healthcare's environmental impact being multidimensional, influenced throughout the healthcare chain, and often has downstream consequences that are hard to identify or to measure. Even though existing research describes many successful approaches to reduce healthcare's environmental impact, a robust multidimensional framework to assess this impact is lacking. This research aims at developing a maturity model for sustainable healthcare that could be used for self-assessment by healthcare professionals to identify improvement actions and for sharing best practices in environmental sustainability., Methods: A design-oriented approach for maturity model development was combined with an expert panel and six case studies to develop, refine and expand the maturity model for environmentally sustainable healthcare., Results: A maturity model was developed containing four domains: 'Governance', 'Organization Structures', 'Processes', and 'Outcomes and Control'. Applying the model in real-world environments demonstrated the model's understandability, ease of use, usefulness, practicality and ability to identify improvement actions for environmental sustainability in healthcare organizations., Conclusions: This study found that healthcare practitioners could apply the maturity model developed and tested in this study in several hours without training to help them gain valuable insights into the environment footprint of the healthcare setting they worked in. Systematically implementing the model developed in this study could help address the urgent need to mitigate the substantial environmental impact of healthcare. These implementations can help evaluate and improve the maturity model., (© 2024. The Author(s).)

Background: Teledermoscopy has demonstrated benefits such as decreased costs and enhanced access to dermatology care for skin cancer detection. However, the heterogeneity among teledermoscopy studies hinders the systematic reviews' synopsis of diagnostic outcomes, impeding trust and adoption in general practice and limiting overall health care benefits., Objective: This study aims to improve understanding and standardization of teledermoscopy diagnostic studies, by identifying and categorizing study characteristics contributing to heterogeneity. Subsequently, the variability and consistency of these characteristics were assessed., Methods: A review of systematic reviews regarding the diagnostic outcomes of teledermoscopy was performed to discern reported study characteristics contributing to heterogeneity. These characteristics were thematically grouped into 3 domains (population, index test, and reference standard), forming a data extraction framework. A scoping review on teledermoscopy diagnostic outcomes studies was performed, guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Data pertaining to study characteristics from included studies were extracted and analyzed through descriptive content analysis. Systematic reviews' reference lists validated the scoping review query., Results: The literature search yielded 4 systematic reviews, revealing 15 heterogeneous studies across the population, index test, and reference standard domains. The scoping review identified 49 studies, with 27 overlapping with the systematic reviews. Population characteristics varied, with one-third (16/49, 33%) of studies reporting fewer than 100 samples; most studies (41/49, 84%) reported on the type of lesion, and most (20/49, 41%) teledermoscopy consultations took place in secondary care. One-fifth (11/49, 22%) did not describe inclusion or exclusion criteria, or the criteria varied highly. Index test characteristics showed differences in clinical expertise, profession, and training in dermatoscopic photography, and 59% (29/49) did not report on 1 or more index test characteristics. Image quality and clinical information reporting likewise varied. Reference standard characteristics involved teledermatologists' assessment, but 16 studies did not report teledermatologists' experience levels. Most studies (26/49, 53%) used histopathology as a gold standard., Conclusions: The heterogeneity in the population, index tests, and reference standard domains across teledermoscopy diagnostic outcome studies underscores the need for standardized reporting. This hinders the synopsis of teledermoscopy diagnostic outcomes in systematic reviews and limits the integration of research results into practice. Adopting a (tailored) STARD (Standards for Reporting Diagnostic Accuracy Studies) checklist for teledermoscopy diagnostic outcome studies is recommended to enhance the consistency and comparability of outcomes. We suggest performing a Delphi study to gather consensus on the tailored STARD guideline., (©Femke van Sinderen, Anne P Langermans, Andre W Kushniruk, Elizabeth M Borycki, Monique M Jaspers, Linda W Peute. Originally published in JMIR Dermatology (http://derma.jmir.org), 18.10.2024.)

Digital health can enhance self-management of patients such as usage of medication reminders, thereby improving health outcomes. However, for successful implementation of such interventions, integration with the electronic health record (EHR) is useful. We evaluated the implementation of an integrated patient portal medication reminder tool in kidney transplant patients. Overall, 40.5% of the patients agreed that integrated EHR medication reminders assisted them in taking their medication on time.

The MOLD-US framework has been developed to synthesize knowledge for (usability) researchers on aging- and disease-related barriers that can hamper the use of health information technology (HIT). However, dissemination in terms of practical applications of the framework is currently unknown and could inform industry and researchers for applying MOLD-US in practice, but also provide insights on the use of theoretical frameworks in HIT research. Therefore, a citation analysis was conducted on the paper presenting the MOLD-US framework. Nine of the 241 citations were found to report practical application(s) of the MOLD-US framework in their methods section: (1) qualitative research input (n=3), (2) research design (n=3), (3) design approaches (n=2), and (4) conceptual framework development (n=2). Future work aims to explore MOLDUS-US practical applications in the industry, through for example grey literature, but also continuously monitor novel applications to enhance the development of HIT for the aging population.

Introduction: Teledermoscopy supports and enhances the early detection of skin lesions by general practitioners in primary practice through remote consultation with dermatologists. Teledermoscopy has been a regular health service in Dutch primary care since February 2009. Teledermoscopy quality and performance outcomes on Dutch general practitioner primary care 11 years after its implementation were assessed., Methods: Dutch primary care teledermoscopy consultation data was retrospectively analysed on timestamps and responses on non-mandatory evaluation questions posed to the general practitioner and teledermatologist during the teledermoscopy consultation process. Anonymized data (February 2009-February 2020) was extracted from a Dutch teledermoscopy service database. The timestamps and evaluation questions data were subject to the teledermoscopy quality and performance outcomes. A limited cost evaluation was performed., Results: A total of 18,738 teledermoscopy consultations were sent by 1341 general practitioners (February 2009-February 2020). For 3908 (31.9%) teledermoscopy consultations, the general practitioner requested second opinion advice which led to 712 (18.2%) extra teledermoscopy referrals of patients who would not have been referred without teledermoscopy, including skin cancer teledermoscopy diagnoses. The general practitioner followed the teledermatologists' advice on patient referral for 8813 (88.5%) patients, reported 97.3% of the teledermoscopy consultations as helpful and 95.1% as instructive, referred 68.0% less patients with teledermoscopy availability, referred overall 59.4% less patients, and needed 5.4 minutes (median) for sending a teledermoscopy consultation. Teledermatologist's median answer and response time was 2 minutes and 2.4 hours, respectively. The estimated cost reduction was €144.18 ($164.65) (50.2%) per teledermoscopy patient., Discussion: Teledermoscopy is a useful service in general practitioner practice for requesting dermatologist advice in primary care settings to support the detection of skin lesion at an early stage and at lower costs. Teledermoscopy could also decrease the burden of secondary dermatology care since general practitioners reported that they did not refer the majority of patients to a dermatologist after the teledermoscopy consultation compared to their initial referral decision. General practitioners reported the teledermoscopy system as helpful and instructive which could contribute to enhancement of their dermatological knowledge., Competing Interests: Declaration of conflicting interestsThe authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: E.T. and F.V.S are employed (part-time) by Ksyos. The remaining authors state no conflicts of interest.

Background: A "Menstruatie Educatie Kalender" application (Menstruation Education Calendar, (MEK-APP)) was developed for adults to evaluate menstrual complaints. The future aim of this app is to use it as a self-diagnostic instrument for menstrual abnormalities for both adults and adolescents. Early identification of the potential of an application for future use by both user groups would increase implementation success and adoption of the application., Objective: To compare differences in experienced usability by adults versus adolescents and to identify factors influencing future use for both age groups in one mHealth application (in this study the MEK-APP)., Methods: This study consisted of three phases: (1) usability testing of the MEK-APP for iOS and Android by think-aloud method, (2) two-month daily use and (3) in-depth individual interview. During the think-aloud sessions, twelve tasks were performed in the application while they were thinking aloud. Usability problems were rated for their severity with Nielsen' Severity Scale. Both the think-aloud sessions and in-depth interviews were verbatim transcribed and thematically analyzed to determine the factors influencing future use for both groups. In addition, the System Usability Scale (SUS) and Intrinsic Motivation Inventory (IMI) questionnaires were filled out during the interviews., Results: Seven adults (>18 years) and seven adolescents (14-18 years) evaluated the MEK-APP. There were 14 usability issues and 16 bugs in both groups. There were no differences between adults and adolescents. In the thematic qualitative analysis, the following future use factors were identified: user-expectation, motivation, privacy, understandability, and user-experience. The user-expectation, motivation and privacy differed between both groups but did not influence usage. No differences were observed in SUS and IMI scores between both groups., Conclusions: There are five factors influencing the future use of a menstrual-related mHealth application for both adults and adolescents. It is possible to serve different age groups with a single application., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)

Introduction: With wider adoption of store-and-forward telemedicine accelerating, particularly post-coronavirus disease 2019, it is essential to understand health care providers' (HCPs) satisfaction with digital solutions offered by telemedicine organizations to (continuously) guarantee service quality. We developed the Store-and-Forward Telemedicine Service User-satisfaction Questionnaire to monitor and assess HCPs' experiences with contracted telemedicine organizations., Methods: Questionnaire construction (phase 1) consisted of exploratory literature search on validated telemedicine satisfaction questionnaires, a telemedicine domain and human factors expert focus group, stakeholder focus group (customer service employee and telemedicine account managers), and two pre-testing rounds among 18 HCPs. The pilot questionnaire (phase 2) was sent to 2179 HCPs for validity and reliability assessment., Results: Phase 1: Two validated questionnaires (73 items overall) were used as input for Store-and-Forward Telemedicine Service User-satisfaction Questionnaire. Revisions resulted in 61 items. Phase 2: the pilot 61-item Store-and-Forward Telemedicine Service User-satisfaction Questionnaire instrument was completed by 181 of 2179 invited HCPs. Forty-one mandatory items of the pilot Store-and-Forward Telemedicine Service User-satisfaction Questionnaire rated on a 5-point Likert scale were included in psychometric analyses and resulted in six reliable scales: training, communication, organization policy and strategy, interaction platform, usage platform, and working conditions., Discussion: The Store-and-Forward Telemedicine Service User-satisfaction Questionnaire is a reliable and valid questionnaire for measuring HCPs' satisfaction with store-and-forward telemedicine services as part of a continuous quality improvement cycle. Reimbursement questions were excluded due to low response. As adoption of telemedicine may be impeded by financial compensation issues, this requires consideration in future telemedicine questionnaires. Store-and-Forward Telemedicine Service User-satisfaction Questionnaire including video consultation items is needed to monitor also synchronous services as these expanded in the coronavirus disease 2019 pandemic., Competing Interests: Declaration of conflicting interestsThe authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: ET is employed by Ksyos and LW is the founder, (co)owner and director of Ksyos. The remaining authors state no conflicts of interest.

Background: The COVID-19 pandemic affected the delivery of primary care and stimulated the use of digital health solutions such as remote digital dermatology care. In the Netherlands, remote store-and-forward dermatology care was already integrated into Dutch general practice before the COVID-19 pandemic. However, it is unclear how general practitioners (GPs) experienced this existing digital dermatology care during the pandemic period., Objective: We investigated GPs' perspectives about facilitators and barriers related to store-and-forward digital dermatology care during the COVID-19 pandemic in the Netherlands, using a sociotechnical approach., Methods: In December 2021, a web-based questionnaire was distributed via email to approximately 3257 GPs who could perform a digital dermatology consultation and who had started a digital consultation (not necessarily dermatology) in the previous 2 years. The questionnaire consisted of general background questions, questions from a previously validated telemedicine service user satisfaction questionnaire, and newly added questions related to the pandemic and use of the digital dermatology service in general practice. The open-ended and free-text responses were analyzed for facilitators and barriers using content analysis, guided by an 8-dimensional sociotechnical model., Results: In total, 71 GPs completed the entire questionnaire, and 66 (93%) questionnaires were included in the data analysis. During the questionnaire distribution period, another national lockdown, social distancing, and stay-at-home mandates were announced; thus, GPs may have had increased workload and limited time to complete the questionnaire. Of the 66 responding GPs, 36 (55%) were female, 25 (38%) were aged 35-44 years, 33 (50%) were weekly platform users, 34 (52%) were working with the telemedicine organization for >5 years, 42 (64%) reported that they used the store-and-forward platform as often during as before the pandemic, 61 (92%) would use the platform again, 53 (80%) would recommend the platform to a colleague, and 10 (15%) used digital dermatology home consultation. Although GPs were generally satisfied with the digital dermatology service, platform, and telemedicine organization, they also experienced crucial barriers to the use of the service during the pandemic. These barriers were GPs' and patients' limited digital photography skills, costs and the lack of appropriate equipment, human-computer interface and interoperability issues on the telemedicine platform, and different use procedures of the digital dermatology service., Conclusions: Although remote dermatology care was already integrated into Dutch GP practice before the pandemic, which may have facilitated the positive responses of GPs about the use of the service, barriers impeded the full potential of its use during the pandemic. Training is needed to improve the use of equipment and quality of (dermoscopy) images taken by GPs and to inform GPs in which circumstances they can or cannot use digital dermatology. Furthermore, the dermatology platform should be improved to also guide patients in taking photographs with sufficient quality., (©Esmée Tensen, Craig Kuziemsky, Monique W Jaspers, Linda W Peute. Originally published in JMIR Dermatology (http://derma.jmir.org), 13.06.2023.)

International student exchange is a valuable opportunity for Biomedical and Health Informatics students to gain new perspectives and experiences. In the past, such exchanges have been made possible through international partnerships between universities. Unfortunately, numerous obstacles such as housing, financial concerns, and environmental implications related to travel, have made it difficult to continue international exchange. Experiences with hybrid and online education during covid-19 paved the way for a new approach that allows for short international exchange with a hybrid online-offline supervision model. This will be initiated with an exploration project between two international universities , each related to their respective institute's research focus.

Objective: The aim of this study was to develop a theory-based framework to enhance and accelerate development, selection, and implementation of solutions mitigating the climate impact of healthcare organizations., Materials and Methods: Existing frameworks were combined to develop the Green-MIssion (Medical Informatics Solutions) framework. It was further developed and refined by mapping solutions from project plans and reviewing it with an expert panel., Results: The framework classifies solutions into three categories: (1) monitor and measure environmental impact of a healthcare setting; (2) help create and increase awareness among employees and patients; and (3) interventions to reduce environmental impacts., Discussion and Conclusion: The framework combines concepts from healthcare information technology and environmental sciences and can be used to structure green medical informatics solutions for different healthcare settings. Furthermore, research should evaluate its application for measuring and assessing the impact of green medical informatics solutions on environmental sustainability and climate resilience., (© The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association.)

Introduction: Older adults living with Alzheimer's disease and related dementias (ADRD) can benefit from mHealth innovations in (daily) care. However, successful implementation and adoption of such innovations can be hindered by a lack of inclusive design. Inclusive design can be challenging, due to the variety of ADRD- and aging-related symptoms that can pose barriers to using mHealth. Previously, a literature-based model with 53 barriers to mHealth use for this population has been developed ("MHealth for OLder adults living with DEMentia - USability" or MOLDEM-US). In this study, we aim to prioritize these through a Delphi study with ADRD experts (case managers, informal caregivers, hospital healthcare professionals, district nurses, and researchers)., Methods: In the first round, participant characteristics and potentially new insights into barriers to mHealth use for older adults living with ADRD were gathered. The consensus questionnaire was submitted in the second round, containing barriers to mHealth use for this population (from MOLDEM-US) with questions inquiring its impact and frequency. In the third round, participants rejudged those barriers for which no consensus (<51 %) or minor consensus (51 % - 60 %) was reached., Results: Thirty-seven participants completed the three rounds of the study. Consensus was reached for eleven barriers after the second round, all having major impact and frequency: integration of functions during daily activities, perceived complexity, efficiency in seeing benefits, trust in own ability, restlessness and agitation, computer literacy, self confidence in using wearables, learnability, working memory, and visual acuity., Conclusion: After round three, consensus was achieved for all 53 barriers. Twenty-six barriers are considered to majorly affect mHealth use, most of which relate to cognition and frame of mind. This study contributes to the development of mHealth design guidelines that take into account the progressive and diverse ADRD- and aging-related symptoms negatively affecting mHealth implementation and adoption., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 The Authors. Published by Elsevier B.V. All rights reserved.)

Background: Challenges remain for general practitioners (GPs) in diagnosing (pre)malignant and benign skin lesions. Teledermoscopy (TDsc) supports GPs in diagnosing these skin lesions guided by teledermatologists' (TDs) diagnosis and advice and prevents unnecessary referrals to dermatology care. However, the impact of the availability of TDsc on GPs' self-reported referral decisions to dermatology care before and after the TDsc consultation is unknown., Objective: The objective of this study is to assess and compare the initial self-reported referral decisions of GPs before TDsc versus their final self-reported referral decisions after TDsc for skin lesions diagnosed by the TD as (pre)malignant or benign., Methods: TDsc consultations requested by GPs in daily practice between July 2015 and June 2020 with a TD assessment and diagnosis were extracted from a nationwide Dutch telemedicine database. Based on GP self-administered questions, the GPs' referral decisions before and their final referral decision after TDsc consultation were assessed for (pre)malignant and benign TD diagnoses., Results: GP self-administered questions and TD diagnoses were evaluated for 6364 TDsc consultations (9.3% malignant, 8.8% premalignant, and 81.9% benign skin lesions). In half of the TDsc consultations, GPs adjusted their initial referral decision after TD advice and TD diagnosis. Initially, GPs did not have the intention to refer 67 (56.8%) of 118 patients with a malignant TD diagnosis and 26 (16.0%) of 162 patients with a premalignant TD diagnosis but then decided to refer these patients after the TDsc consultation. Furthermore, GPs adjusted their decision from referral to nonreferral for 2534 (74.9%) benign skin lesions (including 676 seborrheic keratosis and 131 vascular lesions)., Conclusions: GPs adjusted their referral decision in 52% (n=3306) of the TDsc consultations after the TD assessment. The availability of TDsc is thus of added value and assists GPs in their (non)referral for patients with skin lesions to dermatology care. TDsc resulted in referrals of patients with (pre)malignant skin lesions that GPs would not have referred directly to the dermatologist. TDsc also led to a reduction of unnecessary referrals of patients with low complex benign skin lesions (eg, seborrheic keratosis and vascular lesions)., (©Esmée Tensen, Femke van Sinderen, Marcel W Bekkenk, Monique W Jaspers, Linda W Peute. Originally published in JMIR Dermatology (http://derma.jmir.org), 23.09.2022.)

Objectives: Involving representative users in usability testing of health information technology (HIT) is central to user-centered design. However, (vulnerable) older adults as representative users have unique requirements. Aging processes may affect physical capabilities and cognitive skills, which can hamper testing with this demographic and may require special attention and revised protocols. This study was performed to provide expert-based recommendations for HIT user-testing with (vulnerable) older adults to support inclusive HIT design and evaluation., Methods: First, we conducted a structured workshop with ten experts in HIT implementation and research, recruited through purposeful sampling, to generate insights into how characteristics of older adults may influence user-testing. Next, five Human Factor researchers experienced in HIT user-testing with (vulnerable) older adults validated the results and provided additional textual insights to gain consensus on the most important recommendations. A thematic analysis was performed on the resulting inquiries. Applied codes were based on the User-Centered Design framework., Results: The analysis resulted in nine recommendations for user-testing of HIT with older adults, divided into three main themes: (1) empathetic approach and trust-building, (2) new requirements for testing and study design, and (3) adjustments to usability evaluation methods. For each theme a checklist of relevant items to follow-up on the recommendation is provided., Conclusions: The recommendations generated through expert inquiry contribute to more effective usability testing of HIT with older adults. This provides an important step towards improved accessibility of HIT amongst older adults through inclusive user-centered design., Competing Interests: Disclosure The authors report no conflicts of interest in this work., (IMIA and Thieme. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).)

The availability of mHealth for people with dementia is increasing. Various mHealth design guidelines for this population have been proposed. In this study, we developed a binary checklist with evaluation statements to assess the implementation of twenty design suggestions in seven currently available mHealth apps for people with dementia. Between 17%-65% of the evaluation statements in the checklist were implemented in these apps. Not all statements were considered applicable for each assessed mHealth app, which resulted in dividing the criteria in two groups as either key evaluation statements or optional evaluation statements. In future work we want to augment this checklist to contribute to the future design of mHealth for people with dementia.

mHealth use for people with dementia is fraught with factors influencing its implementation in care and daily life. A better understanding of these factors may provide guidelines to inclusive design. This study aimed to assess whether factors gathered in a literature-based model could be validated by opinions of experts. On basis of a questionnaire as part of a larger study, experts identified barriers that they considered to be related to aging and dementia influencing mHealth use. Nineteen barriers that were mentioned by the dementia experts were covered in our literature-based model. No adaptions to the model were required. The dementia experts acclaimed three barriers to mHealth use that could not be mapped onto the framework: the unavailability of (informal) caregivers to support the mHealth use, the stage and type of dementia of an mHealth user, and the fear of the unknown. These should be considered as prerequisites in the implementation phase of mHealth and explored more in future research.

Background: The number of older adults with Alzheimer's disease and related dementias (ADRD) is increasing worldwide. This offers ample opportunities for mobile health (mHealth) apps, for example to support them in performing daily activities or monitoring their health status. However, how to design these apps taking into consideration ADRD related barriers remains a challenge., Objective: To provide a synthesis of mHealth usability barriers of older adults with ADRD and design suggestions to enhance user-friendliness of mHealth apps., Methods: We performed a scoping review of mHealth studies focused on older adults with ADRD and: (1) usability barriers experienced, and/or (2) implementation or design suggestions. PubMed, Medline, EMBASE, PsycINFO, and Web of Science were searched. Clinical and grey literature on ADRD diminishments was explored to identify potential barriers to using mHealth. Detected usability barriers caused by ADRD impairments were mapped onto the MOLD-US framework through validation, calibration, and adaption. MOLD-US was initially developed to provide an overview of barriers influencing mHealth usability for the general aging population., Results: Title and abstract of 792 unique citations were scanned of which 69 were included for full text review. Fifteen studies matched inclusion criteria. In total 42 barriers were identified to influence mobile health use for older adults with ADRD. Twenty design suggestions were extracted from the studies., Conclusions: The identified usability barriers were classified in five categories: cognition, perception, physical ability, frame of mind, and speech- and language. In addition, the design suggestions were categorized as evidence- or expert-based. Evidence-based design suggestions include showing limited information, repeating instructions multiple times and breaking instructions into simple steps given one at a time. This research provides a first step for further collaboration between ADRD experts and designers to support the development of effective mHealth apps with high user-friendliness., (Copyright © 2021 The Author(s). Published by Elsevier B.V. All rights reserved.)

Objective: Human and Organizational Factors (HOF) studies in health technology involve human beings and thus require Institutional Review Board (IRB) approval. Yet HOF studies have specific constraints and methods that may not fit standard regulations and IRB practices. Gaining IRB approval may pose difficulties for HOF researchers. This paper aims to provide a first overview of HOF study challenges to get IRB review by exploring differences and best practices across different countries., Methods: HOF researchers were contacted by email to provide a testimony about their experience with IRB review and approval. Testimonies were thematically analyzed and synthesized to identify and discuss shared themes., Results: Researchers from seven European countries, Argentina, Canada, Australia, and the United States answered the call. Four themes emerged that indicate shared challenges in legislation, IRB inefficiencies and inconsistencies, general regulation and costs, and lack of HOF study knowledge by IRB members. We propose a model for IRB review of HOF studies based on best practices., Conclusion: International criteria are needed that define low and high-risk HOF studies, to allow identification of studies that can undergo an expedited (or exempted) process from those that need full IRB review. Enhancing IRB processes in such a way would be beneficial to the conduct of HOF studies. Greater knowledge and promotion of HOF methods and evidence-based HOF study designs may support the evolving discipline. Based on these insights, training and guidance to IRB members may be developed to support them in ensuring that appropriate ethical issues for HOF studies are considered., Competing Interests: Disclosure The authors report no conflicts of interest in this work., (Georg Thieme Verlag KG Stuttgart.)

Background: A complex process like the blood transfusion chain could benefit from modern technologies such as radio frequency identification (RFID). RFID could, for example, play an important role in generating logistic and temperature data of blood products, which are important in assessing the quality of the logistic process of blood transfusions and the product itself., Objective: This study aimed to evaluate whether location, time stamp, and temperature data generated in real time by an active RFID system containing temperature sensors attached to red blood cell (RBC) products can be used to assess the compliance of the management of RBCs to 4 intrahospital European and Dutch guidelines prescribing logistic and temperature constraints in an academic hospital setting., Methods: An RFID infrastructure supported the tracking and tracing of 243 tagged RBCs in a clinical setting inside the hospital at the blood transfusion laboratory, the operating room complex, and the intensive care unit within the Academic Medical Center, a large academic hospital in Amsterdam, the Netherlands. The compliance of the management of 182 out of the 243 tagged RBCs could be assessed on their adherence to the following guidelines on intrahospital storage, transport, and distribution: (1) RBCs must be preserved within an environment with a temperature between 2°C and 6°C; (2) RBCs have to be transfused within 1 hour after they have left a validated cooling system; (3) RBCs that have reached a temperature above 10°C must not be restored or must be transfused within 24 hours or else be destroyed; (4) unused RBCs are to be returned to the BTL within 24 hours after they left the transfusion laboratory., Results: In total, 4 blood products (4/182 compliant; 2.2%) complied to all applicable guidelines. Moreover, 15 blood products (15/182 not compliant to 1 out of several guidelines; 8.2%) were not compliant to one of the guidelines of either 2 or 3 relevant guidelines. Finally, 148 blood products (148/182 not compliant to 2 guidelines; 81.3%) were not compliant to 2 out of the 3 relevant guidelines., Conclusions: The results point out the possibilities of using RFID technology to assess the quality of the blood transfusion chain itself inside a hospital setting in reference to intrahospital guidelines concerning the storage, transport, and distribution conditions of RBCs. This study shows the potentials of RFID in identifying potential bottlenecks in hospital organizations' processes by use of objective data, which are to be tackled in process redesign efforts. The effect of these efforts can subsequently be evaluated by the use of RFID again. As such, RFID can play a significant role in optimization of the quality of the blood transfusion chain., (©Linda W Dusseljee-Peute, Remko Van der Togt, Bas Jansen, Monique W Jaspers. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 05.08.2019.)

Background: With populations aging, digital health tools and mobile health applications (mHealth) are becoming more common to assist older people in independent living and self-management of (chronic) illnesses. These mHealth services can be beneficial to older patients, provided that they are adjusted to their needs and characteristics, as the current mHealth landscape lacks user-friendly services for this target group. Understanding of intrinsic aging barriers, which cause and impact usability problems older patients encounter, is needed to achieve this., Objectives: This study set out to assess usability problems older patients encounter in two mHealth apps and aims to show the value of MOLD-US, a recent aging barriers framework, as a classification tool to identify the intrinsic cause of these problems., Method: A case-study design, with in-depth analysis of usability issues older adult patients' experience. Data on usability issues were collected using the Think Aloud Protocol for two mHealth apps. The MOLD-US framework and Nielsen's severity rating were used to classify identified issues and their potential impact., Results: In total 28 high severe usability issues of the mHealth apps were identified. Core natures of most issues were related to motivational and cognitive barriers of older adults. Participants had difficulties in understanding the navigation structure of the apps. Important text, buttons and icon elements were overseen., Conclusion: Current knowledge on creating interfaces for older target groups is not well applied within the assessed mHealth designs. Specifically, design guidelines should address older adults' diminishing cognition skills, physical ability and motivational barriers. By classifying usability problems with MOLD-US, insights on these barriers can be enhanced to adequately address these issues in new designs. In addition, we propose that future research focuses on investigating suitable usability evaluation methods adapted to older patients' characteristics to ultimately be able to gain unbiased sight on usability issues older patients may experience while interacting with technology., (Copyright © 2019 Elsevier B.V. All rights reserved.)

eHealth can improve healthcare worldwide, and scientific research should provide evidence on the efficacy, safety and added value of such interventions. For successful implementation of eHealth interventions into clinical practice, barriers need to be anticipated. We identified seven barriers by interviewing health professionals in the Dutch healthcare system. These barriers covered three topics: financing, human factors and organizational factors. This paper discusses their potential impact on eHealth uptake. Bridging the gap between studies to assess effective eHealth interventions and their value-based implementation in healthcare is much needed.

Objective: To provide an integrated and differentiated understanding of factors influencing guideline-based CDSS implementation and illustrate the gaps in the current literature., Materials and Methods: A systematic literature review and in-depth exploration of factors impeding or facilitating successful implementation of guideline-based CDSS supporting physicians in clinical decision-making was performed. Factors were identified thematically by textual analysis of the included publications and were individually mapped to the human, organization and technology-fit (HOT-fit) framework for evaluating implementations of health information systems., Results: A total of 421 factors were found in 35 included publications from a total of 3676 publications. The mapping of factors concerning CDSS implementation on the HOT-fit framework revealed gaps in each domain of the framework and showed that research has mainly focused on human and technology factors and less on organizational factors., Conclusions: Future research within the field of guideline-based CDSS should focus on evaluating implementations through the use of socio-technical models to study guideline-based CDSS system implementations from a multidimensional view. Furthermore, research is needed to explore whether use of these models during the planning phases of a CDSS project is useful in anticipating and preventing implementation barriers from occurring and exploiting facilitators to a successful implementation of the system., (Copyright © 2016 Elsevier Ireland Ltd. All rights reserved.)

Objectives: To examine the evidence of the impact of patient- centered eHealth applications on patient care and to analyze if and how reported human factor issues mediated the outcomes., Methods: We searched PubMed (2014-2015) for studies evaluating the impact of patient-centered eHealth applications on patient care (behavior change, self-efficacy, and patient health-related outcomes). The Systems Engineering Initiative for Patient Safety (SEIPS 2.0) model was used as a guidance framework to identify the reported human factors possibly impacting the effectiveness of an eHealth intervention., Results: Of the 348 potentially relevant papers, 10 papers were included for data analysis. None of the 10 papers reported a negative impact of the eHealth intervention. Seven papers involved a randomized controlled trial (RCT) study. Six of these RCTs reported a positive impact of the eHealth intervention on patient care. All 10 papers reported on human factor issues possibly mediating effects of patient-centered eHealth. Human factors involved patient characteristics, perceived social support, and (type of) interaction between patient and provider., Conclusion: While the amount of patient-centered eHealth interventions increases, many questions remain as to whether and to what extent human factors mediate their use and impact. Future research should adopt a formal theory-driven approach towards human factors when investigating those factors' influence on the effectiveness of these interventions. Insights could then be used to better tailor the content and design of eHealth solutions according to patient user profiles, so as to enhance eHealth interventions impact on patient behavior, self-efficacy, and health-related outcomes.

Introduction: Cumbersome electronic patient record (EPR) interfaces may complicate data-entry in clinical practice. Completeness of data entered in the EPR determines, among other things, the value of computerized clinical decision support (CCDS). Quantitative usability evaluations can provide insight into mismatches between the system design model of data entry and users' data entry behavior, but not into the underlying causes for these mismatches. Mixed method usability evaluation studies may provide these insights, and thus support generating redesign recommendations for improving an EPR system's data entry interface., Aim: To improve the usability of the data entry interface of an EPR system with CCDS in the field of cardiac rehabilitation (CR), and additionally, to assess the value of a mixed method usability approach in this context., Methods: Seven CR professionals performed a think-aloud usability evaluation both before (beta-version) and after the redesign of the system. Observed usability problems from both evaluations were analyzed and categorized using Zhang et al.'s heuristic principles of good interface design. We combined the think-aloud usability evaluation of the system's beta-version with the measurement of a new usability construct: users' deviations in action sequence from the system's predefined data entry order sequence. Recommendations for redesign were implemented. We assessed whether the redesign improved CR professionals' (1) task efficacy (with respect to the completeness of data they collected), and (2) task efficiency (with respect to the average number of mouse clicks they needed to complete data entry subtasks)., Results: With the system's beta version, 40% of health care professionals' navigation actions through the system deviated from the predefined next system action. The causes for these deviations as revealed by the think-aloud method mostly concerned mismatches between the system design model for data entry action sequences and users expectations of these action sequences, based on their paper-based daily routines. This caused non completion of data entry tasks (31% of main tasks completed), and more navigation actions than minimally required (146% of the minimum required). In the redesigned system the data entry navigational structure was organized in a flexible way around an overview screen to better mimic users' paper-based daily routines of collecting patient data. This redesign resulted in an increased number of completed main tasks (70%) and a decrease in navigation actions (133% of the minimum required). The think-aloud usability evaluation of the redesigned system showed that remaining problems concerned flexibility (e.g., lack of customization options) and consistency (mainly with layout and position of items on the screen)., Conclusion: The mixed method usability evaluation was supportive in revealing the magnitude and causes of mismatches between the system design model of data-entry with users' data entry behavior. However, as both task efficacy and efficiency were still not optimal with the redesigned EPR, we advise to perform a cognitive analysis on end users' mental processes and behavior patterns in daily work processes specifically during the requirements analysis phase of development of interactive healthcare information systems., (Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.)

Objective: To investigate whether the use of the think-aloud method with propositional analysis could be helpful in the design of a Clinical Decision Support System (CDSS) providing guideline recommendations about long-term follow-up of childhood cancer survivors., Materials and Methods: The think-aloud method was used to gain insight into healthcare professionals' information processing while reviewing a paper-based guideline. A total of 13 healthcare professionals (6 physicians and 7 physician assistants) prepared 2 fictitious patient consults using the paper-based guideline. Propositional analysis was used to analyze verbal protocols of the think-aloud sessions. A prototype CDSS was developed and a usability study was performed, again with the think-aloud method., Results: The analysis revealed that the paper-based guideline did not support healthcare practitioners in finding patient-specific recommendations. An information processing model for retrieving recommendations was developed and used as input for the design of a CDSS prototype user interface. Usability analysis of the prototype CDSS showed that the navigational structure of the system fitted well with healthcare practitioners' daily practices., Conclusions: The think-aloud method combined with propositional analysis of healthcare practitioners' verbal utterances while they processed a paper-based guideline was useful in the design of a usable CDSS providing patient-specific guideline recommendations., (Copyright © 2015 Elsevier Ireland Ltd. All rights reserved.)

This paper discusses the preliminary results of a literature review on studies published in 2014-2015 concerning patient-centered mHealth applications' (apps) impact. Abstracts were included when they described a mHealth app targeted at patients and reported on the effects of this app on patient care. From a total of 559 potentially relevant articles, 17 papers were finally included. Nine studies reported a positive impact of the patient-centered mHealth app on patient care; 4 of these studies were randomized controlled trials. Measured impacts in the 17 studies focused on improving patients' physical activity, self-efficacy and medication adherence. Human factors issues potentially mediating these effects were discussed in all studies. Transitions in the interaction between healthcare providers and their patients were most often discussed as influencing the impact of the mHealth app. More research is needed, focussing on human issues mediating the effect of patient-centered mHealth apps to precipitate knowledge on the effectiveness of mHealth. This research should preferably be guided by socio-technical models.

Background: The Website Developmental Model for the Healthcare Consumer (WDMHC) is an extensive and successfully evaluated framework that incorporates user-centered design principles. However, due to its extensiveness its application is limited. In the current study we apply a subset of the WDMHC framework in a case study concerning the development and evaluation of a website aimed at childhood cancer survivors (CCS)., Objective: To assess whether the implementation of a limited subset of the WDMHC-framework is sufficient to deliver a high-quality website with few usability problems, aimed at a specific patient population., Methods: The website was developed using a six-step approach divided into three phases derived from the WDMHC: 1) information needs analysis, mock-up creation and focus group discussion; 2) website prototype development; and 3) heuristic evaluation (HE) and think aloud analysis (TA). The HE was performed by three double experts (knowledgeable both in usability engineering and childhood cancer survivorship), who assessed the site using the Nielsen heuristics. Eight end-users were invited to complete three scenarios covering all functionality of the website by TA., Results: The HE and TA were performed concurrently on the website prototype. The HE resulted in 29 unique usability issues; the end-users performing the TA encountered eleven unique problems. Four issues specifically revealed by HE concerned cosmetic design flaws, whereas two problems revealed by TA were related to website content., Conclusion: Based on the subset of the WDMHC framework we were able to deliver a website that closely matched the expectancy of the end-users and resulted in relatively few usability problems during end-user testing. With the successful application of this subset of the WDMHC, we provide developers with a clear and easily applicable framework for the development of healthcare websites with high usability aimed at specific medical populations.

Objective: To compare the performance of the Concurrent (CTA) and Retrospective (RTA) Think Aloud method and to assess their value in a formative usability evaluation of an Intensive Care Registry-physician data query tool designed to support ICU quality improvement processes., Methods: Sixteen representative intensive care physicians participated in the usability evaluation study. Subjects were allocated to either the CTA or RTA method by a matched randomized design. Each subject performed six usability-testing tasks of varying complexity in the query tool in a real-working context. Methods were compared with regard to number and type of problems detected. Verbal protocols of CTA and RTA were analyzed in depth to assess differences in verbal output. Standardized measures were applied to assess thoroughness in usability problem detection weighted per problem severity level and method overall effectiveness in detecting usability problems with regard to the time subjects spent per method., Results: The usability evaluation of the data query tool revealed a total of 43 unique usability problems that the intensive care physicians encountered. CTA detected unique usability problems with regard to graphics/symbols, navigation issues, error messages, and the organization of information on the query tool's screens. RTA detected unique issues concerning system match with subjects' language and applied terminology. The in-depth verbal protocol analysis of CTA provided information on intensive care physicians' query design strategies. Overall, CTA performed significantly better than RTA in detecting usability problems. CTA usability problem detection effectiveness was 0.80 vs. 0.62 (p<0.05) respectively, with an average difference of 42% less time spent per subject compared to RTA. In addition, CTA was more thorough in detecting usability problems of a moderate (0.85 vs. 0.7) and severe nature (0.71 vs. 0.57)., Conclusion: In this study, the CTA is more effective in usability-problem detection and provided clarification of intensive care physician query design strategies to inform redesign of the query tool. However, CTA does not outperform RTA. The RTA additionally elucidated unique usability problems and new user requirements. Based on the results of this study, we recommend the use of CTA in formative usability evaluation studies of health information technology. However, we recommend further research on the application of RTA in usability studies with regard to user expertise and experience when focusing on user profile customized (re)design., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Mobile health (mHealth) apps can play a significant role in supporting older patients. The aging process is yet related to progressive degradation of sensory, physical and cognitive abilities. mHealth apps with touch screen based interfaces thus need to be adapted to the, often impaired, skills and special needs of older patients. However, existing design guidelines for mHealth apps do not address the aging barriers that older users face in working with touch screen based applications. By means of a constructed conceptual framework on cognitive, motivational, physical and visual impairments of older adults, this study revealed possible aging barriers concerning identified usability issues of an mHealth app for older patients. Insights into the aging process barriers that underlie usability issues supports developers in (re)designing mHealth apps that are better attuned to the needs of the older patient population.

In a Health Information Technology (HIT) regulatory context in which the usability of this technology is more and more a critical issue, there is an increasing need for evidence based usability practice. However, a clear definition of evidence based usability practice and how to achieve it is still lacking. This paper underlines the need for evidence based HIT design and provides a definition of evidence based usability practice as the conscientious, explicit and judicious use of current best evidence in making decisions in design of interactive systems in health by applying usability engineering and usability design principles that have proven their value in practice. Current issues that hamper evidence based usability practice are highlighted and steps needed to achieve evidence are presented.

Objective: The rising incidence of type 2 diabetes mellitus (T2DM) induces severe challenges for the health care system. Our research group developed a web-based system named PANDIT that provides T2DM patients with insulin dosing advice using state of the art clinical decision support technology. The PANDIT interface resembles a glucose diary and provides advice through pop-up messages. Diabetes nurses (DNs) also have access to the system, allowing them to intervene when needed. The objective of this study was to establish whether T2DM patients can safely use PANDIT at home. To this end, we assessed whether patients experience usability problems with a high risk of compromising patient safety when interacting with the system, and whether PANDIT's insulin dosing advice are clinically safe., Research Design and Methods: The study population consisted of patients with T2DM (aged 18-80) who used a once daily basal insulin as well as DNs from a university hospital. The usability evaluation consisted of think-aloud sessions with four patients and three DNs. Video data, audio data and verbal utterances were analyzed for usability problems encountered during PANDIT interactions. Usability problems were rated by a physician and a usability expert according to their potential impact on patient safety. The usability evaluation was followed by an implementation with a duration of four weeks. This implementation took place at the patients' homes with ten patients to evaluate clinical safety of PANDIT advice. PANDIT advice were systematically compared with DN advice. Deviating advice were evaluated with respect to patient safety by a panel of experienced physicians, which specialized in diabetes care., Results: We detected seventeen unique usability problems, none of which was judged to have a high risk of compromising patient safety. Most usability problems concerned the lay-out of the diary, which did not clearly indicate which data entry fields had to be entered in order to obtain an advice. 27 out of 74 (36.5%) PANDIT advice differed from those provided by DNs. However, only one of these (1.4%) was considered unsafe by the panel., Conclusion: T2DM patients with no prior experience with the web-based self-management system were capable of consulting the system without encountering significant usability problems. Furthermore, the large majority of PANDIT advice were considered clinically safe according to the expert panel. One advice was considered unsafe. This could however easily be corrected by implementing a small modification to the system's knowledge base., (Copyright © 2013 Elsevier B.V. All rights reserved.)

Objective: To assess whether a user-centred prototype clinical decision support system (CDSS) providing patient-specific advice better supports healthcare practitioners in terms of (a) types of usability problems detected and (b) effective and efficient retrieval of childhood cancer survivor's follow-up screening procedures compared to an expert-driven paper-based guideline., Methods and Materials: A user-centred design (UCD) process was employed to design a prototype CDSS. Usability problems in information retrieval with the paper-based guideline were assessed by think-aloud analysis with 13 participants. Both simple and more complex tasks were applied. The analysis provided input for the UCD process of the prototype. The usability of the prototype CDSS was subsequently evaluated by think-aloud analysis with the same participants. Usability problems of the paper-based guideline and the prototype CDSS were compared by using the classification of usability problems scheme. In addition, efficiency (time to complete task) and effectiveness (completeness of retrieved screening procedures) of information retrieval of participants in the expert-driven paper-based guideline and the user-centred prototype CDSS were compared., Results: Usability problems in both the paper-based guideline and the CDSS prototype were mainly classified as 'incongruent with participants' mental model'. The prototype CDSS reduced this type of problem from 17 to 6 problems. The time to perform simple information retrieval tasks increased by 58 s when using the prototype CDSS, however, it resulted in a 58% improvement in task completeness compared to the paper-based guideline. The time to perform complex scenarios decreased by 3:50 min with the prototype CDSS, with 17% higher completeness compared to the paper-based guideline., Conclusion: Analysis showed that usability problems experienced by healthcare practitioners when using a paper-based guideline could be overcome by implementing the guideline in a user-centred CDSS design. Although different types of usability problems were experienced with the prototype CDSS, they did not inhibit effective and efficient performance of tasks in the system. The usability problem analysis of the paper-based guideline effectively supported comparison of usability problems found in the two information retrieval systems and it supported the UCD of the CDSS., (Copyright © 2013 Elsevier B.V. All rights reserved.)

Increasingly, studies are being published on the potential negative effect of introducing poor designed Health Information Technology (HIT) into clinical settings, relating to technology-induced errors and adverse events. Academic research on HIT design and evaluation is an extremely important source of information in providing new insights into factors contributing to successful system (re)design efforts, system user-friendliness and usability issues and safety critical aspects of HIT design. However, these studies have been inconsistent and incomprehensive in their reporting, complicating the appraisal of outcomes, generalizability of study findings, meta-analysis and harmonization of the available evidence. To improve identification of type of use errors and safety related issues regarding design and implementation of HIT, consensus on issues to be reported on in scientific publications is a necessary step forward. This study presents the first approach to a framework providing a set of principles to follow for comprehensive and unambiguous reporting of HIT design and usability evaluation studies with the objective to reduce variation, improve on the publication reporting quality and proper indexation of these studies. This framework may be helpful in expanding the knowledge base not only concerning the application of Human Factors (HF)/Usability studies of HIT but also improve the knowledge base of how to (re)design and implement effective, efficient and safe HIT.

We developed a web-based system supporting patients in insulin self-titration and their caregivers in monitoring patients' self-management activities. Since usability flaws could cause user attrition and compromise patient safety, we evaluated the system's usability prior to its implementation in practice. Two pairs of researchers conducted cognitive walkthrough sessions and identified 81 unique usability problems, including four with a potential impact on patient safety. Usability evaluations could reveal many usability problems and allow solving the problems while avoiding user attrition and potential adverse patient events.

Objective: To synthesize the literature on clinical decision-support systems' (CDSS) impact on healthcare practitioner performance and patient outcomes., Design: Literature search on Medline, Embase, Inspec, Cinahl, Cochrane/Dare and analysis of high-quality systematic reviews (SRs) on CDSS in hospital settings. Two-stage inclusion procedure: (1) selection of publications on predefined inclusion criteria; (2) independent methodological assessment of preincluded SRs by the 11-item measurement tool, AMSTAR. Inclusion of SRs with AMSTAR score 9 or above. SRs were thereafter rated on level of evidence. Each stage was performed by two independent reviewers., Results: 17 out of 35 preincluded SRs were of high methodological quality and further analyzed. Evidence that CDSS significantly impacted practitioner performance was found in 52 out of 91 unique studies of the 16 SRs examining this effect (57%). Only 25 out of 82 unique studies of the 16 SRs reported evidence that CDSS positively impacted patient outcomes (30%)., Conclusions: Few studies have found any benefits on patient outcomes, though many of these have been too small in sample size or too short in time to reveal clinically important effects. There is significant evidence that CDSS can positively impact healthcare providers' performance with drug ordering and preventive care reminder systems as most clear examples. These outcomes may be explained by the fact that these types of CDSS require a minimum of patient data that are largely available before the advice is (to be) generated: at the time clinicians make the decisions.

Clinical Decision Support Systems (CDSS) have been shown to improve clinical performance and patient outcomes, but the failure rate of such systems is still over 50 percent. To contribute to a wider understanding of issues surrounding CDDS acceptance, we performed a systematic review of studies that evaluated CDSS implementations in clinical care to determine the factors that are associated with acceptance of CDSS by physicians. The factors that were found were categorized according to the HOT-fit framework. The mapping of factors concerning CDSS acceptance on the HOT-fit framework revealed gaps in each domain of the framework and showed that research has mainly focused on human and technology factors and a lack of research on organizational factors. A potential area of research could thus be studying the organizational factors that may influence CDSS acceptance.

Interactive Health Information systems are often considered cognitively complex by their users, leading to high cognitive burden and increased workload. This paper explores if Think Aloud usability testing provides valuable input to effectively redesign a web-based Data Query Tool in Intensive Care and to reduce physicians' cognitive workload during system interaction. Pre and post redesign usability testing demonstrated a major reduction in the cognitive task workload after redesign of the tool. Classification of revealed usability problems by means of the User Action Framework pointed out that usability problems related to the cognitively planning of actions by system users foremost affected cognitive task workload. This result may support Health Information system (re)design efforts on how to tackle the system's cognitive complexity and in so doing improve on its usability.

Computerized alerts provided by health care information systems have been shown to enhance clinical practice. However, clinicians still override more than half of the alerts. This indicates that certain aspects of alerts need improvement to fulfill their purpose of supporting clinicians in decision making. This paper reports on a systematic review on studies evaluating alert specifications and their impact on clinicians' alert adherence. The review revealed that use of colors and icons to distinguish different alert severity levels and presenting high severity alerts in an interruptive fashion increases clinicians adherence to alert recommendations. Alert message contents that lack clinical importance or provide incorrect texts increase alert non-adherence. Few studies have yet focused on the impact of alert specifications on clinicians' adherence. A research agenda is needed on alert specifications and their impact on clinicians' adherence in order to develop alerts that truly support clinician decision making.

Objective: To investigate the human, social and organizational issues surrounding a Computerized Physician Order Entry system for Laboratory ordering (CPOE-L) implementation process and to analyze their interrelated effects on the system implementation failure in an academic medical setting. Second, to provide lessons learned and recommendations on to how to manage challenges of human, social and organizational nature surrounding CPOE-L implementations., Methods: The themes surrounding CPOE introduction were identified by a heuristic analysis of literature on CPOE implementations. The resulting set of themes was applied as a reference model for 20 semi-structured interviews conducted during the CPOE-L implementation process with 11 persons involved in the CPOE-L project and in reviewing all CPOE-L related project documentation. Data was additionally gathered by user questionnaires, by user discussion rounds and through an ethnographical study performed at the involved clinical and laboratory departments. In analyzing the interview transcripts, project documentation and data from user questionnaires and discussion rounds a grounded theory approach was applied by the evaluation team to identify problem areas or issues deserving further analysis., Results: Outlined central problem areas concerning the CPOE-L implementation and their mutual relations were depicted in a conceptual interpretative model. Understanding of clinical workflow was identified as a key theme pressured by organizational, human and social issues ultimately influencing the entire implementation process in a negative way. Vast delays in CPOE introduction, system immaturity and under-functionality could all be directly attributed to a superficial understanding of workflow. Consequently, final CPOE integration into clinical and laboratory workflows was inhibited by both end-users as well as department managers and withdrawal of the CPOE-L system became inevitable., Conclusion: This case study demonstrates which human, social and organizational issues relevant to CPOE implementation cumulatively led to a failure outcome of the CPOE-L pilot introduction. The experiences and considerations described in this paper show important issues for CPOE systems to be successfully introduced and to be taken into account in future CPOE implementations. Understanding and consideration of (clinical) workflow aspects by project managers and the involved clinical organization is of extreme importance from the very start of a CPOE implementation process., ((c) 2009 Elsevier Ireland Ltd. All rights reserved.)

This paper discusses the preliminary results of a systematic review of the literature on applied usability studies of health information systems in the period 1990 to 2006. Abstracts were included when they described an evaluation of the usability of a health information system. To gain insight into usability methods applied and their properties we constructed a framework to analyze the studies. The framework includes objectives, designs, number of participants, user-profiles, settings, medical domain, and type of health information systems evaluated. Fifty-two Papers were included in the review. Findings show that from 2002 an increasing trend can be observed of publication of usability studies. Most studies discuss summative usability results on working systems thereby focusing on systems' adoption problems. Formative usability studies lack a uniform way to describe how study results contributed to the system's iterative development cycle.

Physicians' acceptance of Electronic Medical Record Systems (EMRs) is closely related to their usability. Knowledge about end-users' opinions on usability of an EMR system may contribute to planning for the next phase of the usability cycle of the system. A demand for integration of new functionalities, such as computerized order entry and an electronic patient status led to redesign of our EMR system, which had been in use for over 8 years at the Academic Medical Center of Amsterdam. The aim of this study was to understand whether the redesigned EMR system was an improvement of the earlier EMR and which system aspects accounted for user satisfaction and which did not. We conducted a formative pre- and post usability evaluation of our former and redesigned EMR system. For the assessment of both system versions' usability, we distributed two standardized usability questionnaires among 150 clinicians who routinely had used the older EMR system and had been working with its newer version for 6 weeks. Though overall user satisfaction was relatively high for both EMR systems, screen layout and interaction structure proved less easy to work with in the newer EMR system. The new EMR system however was more appreciated because of its enhanced functionality, capabilities and likeable user-interface. The results point to a number of actions that might be useful in future usability improvement efforts of our EMR system and other EMRs.

Objectives: To assess the usability of an emerging POE system, OM/Lab, for the computer-supported ordering of laboratory tests. We were more specifically interested in the relation of the usability problems detected in the user testing sessions with the order behaviour in terms of efficiency and errors in ordering., Methods: A cognitive walkthrough of the OM/Lab system was conducted by two analysts using four real-life scenarios for ordering laboratory tests, which were reviewed for comprehensiveness by an expert clinician. Thereafter, the OM/Lab system was evaluated on its usability in testing sessions with seven potential end-users of the system performing these same four scenarios. The results of these end-user testing sessions were used to analyze the effect of usability flaws on the quality of ordering in terms of omissions, errors in orders and cancelled orders., Results: The analyses revealed a total of 33 usability problems, of which 25 problems were revealed both by the cognitive walkthrough and in the end-user sessions. These 25 usability problems indeed led to inefficient order behaviour, omissions and errors in orders and even to cancelled orders., Discussion: Our results revealed that the OM/Lab system suffered from a high number of usability flaws. The interface design flaws were, among other things, related to misallocation of buttons on the screen, incomprehensibility of button labels and feedback containing no relevant information to the user about the cause of errors made and consequences of a user's action. Additionally, our user test session results indicated that the OM/Lab system also suffered from user interaction problems of a more socio-technical nature. These sessions revealed, among other things, that the more specific action sequences to be executed within the ordering model of the OM/Lab system did not correspond to the daily working routines of end-users and that the grouping of laboratory tests within clusters did not match the paper-based order forms. The seemingly negative effects of these usability flaws on the quality of ordering and the inefficiency of work processes during the pilot implementation finally lead to withdrawal of the OM/Lab system from clinical practice. Though the system implementation failed, our usability study brought research on (re)designing and evaluating clinical computer applications at a higher status in our institution. It is now recognized that usability evaluation studies that will support good quality of clinical practice are highly important.

Though designing physician order entry (POE) systems has been a highly discussed research topic in the last decade, it seems that engineering POE systems that truly optimise the quality and efficiency of ordering is still a challenge. This study addresses a usability evaluation of an emerging POE for the electronically ordering of laboratory tests. By applying two complementary cognitive approaches, specifically the cognitive walkthrough and think aloud method with seven potential end users of the system, we analyzed usability problems in the prototype system and their potential effect on the quality of orders in terms of efficiency and errors in ordering. The cognitive walkthrough provided a coding scheme that was used to analyse in more detail usability errors encountered during the think aloud tests with the seven end users. The analyses revealed a total of 33 usability problems, which indeed led to inefficiency, omissions in ordering and even to cancelled orders. Most of these usability problems referred to incomprehensiveness of required actions by the user and incomprehensiveness of text used in the system. Next to the discussion of the reasons for these usability problems, the surplus value of the think aloud method as supplementary to the cognitive walkthrough in evaluating physician order entry systems is outlined in this paper.