Search

Background: Symptom checkers are digital health applications (smartphone applications or website-based applications) to support laypersons in clinical decision making. Besides providing suggestions on probable diagnoses, symptom checkers appraise the urgency of patient reported medical complaints (triage recommendation). Despite past studies rating the accuracy of symptom checkers as deficient, these apps are becoming increasingly popular among the general public. Until now, no study has evaluated whether symptom checker triage accuracy is superior to that of their intended user group, that is laypersons. Methods: In an online survey, participants had to assess the treatment urgency of 45 fictitious, short patient descriptions (case vignettes). We recruited 91 US participants via the platform Amazon Mechanical Turk. Data on triage accuracy for 15 symptom checkers on the same case vignettes was provided by a previous study. We compared the triage accuracy between symptom checkers and laypersons, for all 45 vignettes and for each of three urgency levels. We further investigated whether laypersons and symptom checkers are inclined towards over-triage or under-triage. In exploratory analyses we searched for effects of age, gender and level education on participants’ triage accuracy and inclination towards over-triaging. Results: On average, participants’ triage accuracy (60.9%; 95% CI 59.5%-62.3%) was similar to that of symptom checkers (58%). The majority of participants outperformed ten out of 15 symptom checkers in terms of overall triage accuracy. Both participants and symptom checkers were inclined towards over-triage rather than under-triage. We detected no influence of socio-demographic variables on participants’ triage accuracy. Female participants had a higher ratio of over-triage to under-triage (2:1) errors than male participants (1.2:1). Discussion: While on average symptom checkers have no superior triage accuracy than laypersons, five symptom checkers outperformed the majority of participants. Whether symptom checker usage is beneficial, depends not only on the symptom checker, but also on the user and the specific use case. Future studies should investigate how symptom checkers can balance out laypersons’ deficits and blind spots while not misleading them when their own intuition proves correct. Future research on when and why laypersons trust symptom checker appraisals will prove valuable., Hintergrund: Symptom Checker Apps sind digitale Anwendungen (Smartphone Apps oder Webseiten) die Laiennutzer bei klinischen Entscheidungen unterstützen. Neben der Einschätzung, welche Diagnosen für ein beschriebenes Beschwerdebild wahrscheinlich seien, geben sie auch oft Empfehlungen, ob und wo ein Nutzer medizinische Hilfe aufsuchen sollte ("Triage Empfehlung"). Obwohl die Genauigkeit von Symptom Checker Apps in unabhängigen Studien bisher eher als unzureichend bewertet wurde, erfreuen sich solche Apps wachsender Beliebtheit. Bisher wurde noch nicht verglichen, ob solche Apps medizinischen Laien bei der Triage-Einschätzung überlegen sind. Methoden: Auf Amazon MTurk haben wir 91 US-amerikanische Probanden rekrutiert. In einer Online-Umfrage schätzten die Probanden die Dringlichkeit von 45 fiktiven, kurzen Fallvignetten ein. Daten zu 15 Symptom Checker Apps, die anhand der gleichen 45 Fallvignetten getestet wurden, wurden aus einer vorigen Studie übernommen. Wir verglichen die Genauigkeit der Triage-Einschätzung zwischen Symptom Checker Apps und den Laien, bezogen auf alle 45 Fallvignetten und pro Dringlichkeitsstufe. Zudem wurde bestimmt, ob die Apps und Laien eher über- oder untertriagieren. Explorativ haben wir analysiert, ob Alter, Geschlecht und Bildungshintergrund einen Einfluss auf die Triage-Genauigkeit und die Neigung zur Übertriage haben bei den Laien haben. Ergebnisse: Im Gesamtdurchschnitt waren die Triage-Genauigkeiten der Probanden (60.9%; 95% KI 59.5%-62.3%) und Symptom Checker Apps (58%) sehr ähnlich. Der Mehrheit der Probanden gelang es, besser als zehn von 15 Symptom Checker Apps zu triagieren. Sowohl die Symptom Checker als auch die Laien machten mehr Übertriage-Fehler als Untertriage-Fehler. Einen Einfluss soziodemographischer Merkmale auf die Triage-Genauigkeit bei den Laien zeigte sich nicht. Das Verhältnis von Übertriage- zu Untertriage-Fehlern war bei Frauen (2:1) höher als bei Männern (1.2:1). Diskussion: Während die meisten Symptom Checker Apps keine höhere Triage-Genauigkeit hatten als der durchschnittliche Proband, gab es fünf Apps, die der deutlichen Mehrheit der Probanden überlegen war. Ob die Verwendung von Symptom Checker Apps nützlich ist, hängt nicht nur ab von der Fähigkeit solcher Apps, sondern auch von denen ihrer Nutzer sowie den spezifischen Anwendungsfall. Weitere Studien sollten untersuchen, wie Symptom Checker Apps die Defizite ihrer Nutzer ausgleichen können, ohne sie fehlzuleiten, wenn die Nutzer in ihrer Einschätzung richtig liegen. Erkenntnisse dazu, in welchen Fällen und warum Nutzer den Einschätzungen von Symptom Checker Apps trauen, werden hierbei sehr wertvoll sein.

Background: Although medical decision-making may be thought of as a task involving health professionals, many decisions, including critical health-related decisions are made by laypersons alone. Specifically, as the first step to most care episodes, it is the patient who determines whether and where to seek health care (triage). Overcautious self-assessments (ie, overtriaging) may lead to overutilization of health care facilities and overcrowded emergency departments, whereas imprudent decisions (ie, undertriaging) constitute a risk to the patient's health. Recently, patient-facing decision support systems, commonly known as symptom checkers, have been developed to assist laypersons in these decisions. Objective: The purpose of this study is to identify factors influencing laypersons' ability to self-triage and their risk averseness in self-triage decisions. Methods: We analyzed publicly available data on 91 laypersons appraising 45 short fictitious patient descriptions (case vignettes; N=4095 appraisals). Using signal detection theory and descriptive and inferential statistics, we explored whether the type of medical decision laypersons face, their confidence in their decision, and sociodemographic factors influence their triage accuracy and the type of errors they make. We distinguished between 2 decisions: whether emergency care was required (decision 1) and whether self-care was sufficient (decision 2). Results: The accuracy of detecting emergencies (decision 1) was higher (mean 82.2%, SD 5.9%) than that of deciding whether any type of medical care is required (decision 2, mean 75.9%, SD 5.25%; t>90=8.4; P89=3.7; P

Background: During the COVID-19 pandemic, medical laypersons with symptoms indicative of a COVID-19 infection commonly sought guidance on whether and where to find medical care. Numerous web-based decision support tools (DSTs) have been developed, both by public and commercial stakeholders, to assist their decision making. Though most of the DSTs’ underlying algorithms are similar and simple decision trees, their mode of presentation differs: some DSTs present a static flowchart, while others are designed as a conversational agent, guiding the user through the decision tree’s nodes step-by-step in an interactive manner. Objective: This study aims to investigate whether interactive DSTs provide greater decision support than noninteractive (ie, static) flowcharts. Methods: We developed mock interfaces for 2 DSTs (1 static, 1 interactive), mimicking patient-facing, freely available DSTs for COVID-19-related self-assessment. Their underlying algorithm was identical and based on the Centers for Disease Control and Prevention’s guidelines. We recruited adult US residents online in November 2020. Participants appraised the appropriate social and care-seeking behavior for 7 fictitious descriptions of patients (case vignettes). Participants in the experimental groups received either the static or the interactive mock DST as support, while the control group appraised the case vignettes unsupported. We determined participants’ accuracy, decision certainty (after deciding), and mental effort to measure the quality of decision support. Participants’ ratings of the DSTs’ usefulness, ease of use, trust, and future intention to use the tools served as measures to analyze differences in participants’ perception of the tools. We used ANOVAs and t tests to assess statistical significance. Results: Our survey yielded 196 responses. The mean number of correct assessments was higher in the intervention groups (interactive DST group: mean 11.71, SD 2.37; static DST group: mean 11.45, SD 2.48) than in the control group (mean 10.17, SD 2.00). Decisional certainty was significantly higher in the experimental groups (interactive DST group: mean 80.7%, SD 14.1%; static DST group: mean 80.5%, SD 15.8%) compared to the control group (mean 65.8%, SD 20.8%). The differences in these measures proved statistically significant in t tests comparing each intervention group with the control group (P