Accuracy assessment of ChatGPT responses to frequently asked questions regarding anterior cruciate ligament surgery.

Background: The emergence of artificial intelligence (AI) has allowed users to have access to large sources of information in a chat-like manner. Thereby, we sought to evaluate ChatGPT-4 response's accuracy to the 10 patient most frequently asked questions (FAQs) regarding anterior cruciate ligament (ACL) surgery.
Methods: A list of the top 10 FAQs pertaining to ACL surgery was created after conducting a search through all Sports Medicine Fellowship Institutions listed on the Arthroscopy Association of North America (AANA) and American Orthopaedic Society of Sports Medicine (AOSSM) websites. A Likert scale was used to grade response accuracy by two sports medicine fellowship-trained surgeons. Cohen's kappa was used to assess inter-rater agreement. Reproducibility of the responses over time was also assessed.
Results: Five of the 10 responses received a 'completely accurate' grade by two-fellowship trained surgeons with three additional replies receiving a 'completely accurate' status by at least one. Moreover, inter-rater reliability accuracy assessment revealed a moderate agreement between fellowship-trained attending physicians (weighted kappa = 0.57, 95% confidence interval 0.15-0.99). Additionally, 80% of the responses were reproducible over time.
Conclusion: ChatGPT can be considered an accurate additional tool to answer general patient questions regarding ACL surgery. None the less, patient-surgeon interaction should not be deferred and must continue to be the driving force for information retrieval. Thus, the general recommendation is to address any questions in the presence of a qualified specialist.
Competing Interests: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: ‘J.C. reports a relationship with American Orthopaedic Society for Sports Medicine that includes board membership, a relationship with Arthrex Inc that includes consulting or advisory, a relationship with Arthroscopy Association of North America that includes board membership, a relationship with CONMED Corp that includes consulting or advisory, a relationship with International Society of Arthroscopy Knee Surgery and Orthopaedic Sports Medicine that includes board membership, a relationship with Ossur Americas that includes consulting or advisory, a relationship with Smith and Nephew Inc that includes consulting or advisory and speaking and lecture fees. All other authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.’.
(Copyright © 2024 Elsevier B.V. All rights reserved.)