Search

Clinical trial matching is the task of identifying trials for which patients may be eligible. Typically, this task is labor-intensive and requires detailed verification of patient electronic health records (EHRs) against the stringent inclusion and exclusion criteria of clinical trials. This process also results in many patients missing out on potential therapeutic options. Recent advancements in Large Language Models (LLMs) have made automating patient-trial matching possible, as shown in multiple concurrent research studies. However, the current approaches are confined to constrained, often synthetic, datasets that do not adequately mirror the complexities encountered in real-world medical data. In this study, we present an end-to-end large-scale empirical evaluation of a clinical trial matching system and validate it using real-world EHRs. We perform comprehensive experiments with proprietary LLMs and our custom fine-tuned model called OncoLLM and show that OncoLLM outperforms GPT-3.5 and matches the performance of qualified medical doctors for clinical trial matching., (© 2024. The Author(s).)

Background: Few studies have examined the performance of artificial intelligence (AI) content detection in scientific writing. This study evaluates the performance of publicly available AI content detectors when applied to both human-written and AI-generated scientific articles., Methods: Articles published in Annals of Surgical Oncology (ASO) during the year 2022, as well as AI-generated articles using OpenAI's ChatGPT, were analyzed by three AI content detectors to assess the probability of AI-generated content. Full manuscripts and their individual sections were evaluated. Group comparisons and trend analyses were conducted by using ANOVA and linear regression. Classification performance was determined using area under the curve (AUC)., Results: A total of 449 original articles met inclusion criteria and were evaluated to determine the likelihood of being generated by AI. Each detector also evaluated 47 AI-generated articles by using titles from ASO articles. Human-written articles had an average probability of being AI-generated of 9.4% with significant differences between the detectors. Only two (0.4%) human-written manuscripts were detected as having a 0% probability of being AI-generated by all three detectors. Completely AI-generated articles were evaluated to have a higher average probability of being AI-generated (43.5%) with a range from 12.0 to 99.9%., Conclusions: This study demonstrates differences in the performance of various AI content detectors with the potential to label human-written articles as AI-generated. Any effort toward implementing AI detectors must include a strategy for continuous evaluation and validation as AI models and detectors rapidly evolve., (© 2024. Society of Surgical Oncology.)

Objectives: While patient interest in telehealth increases, clinicians' perspectives may influence longer-term adoption. We sought to identify facilitators and barriers to continued clinician incorporation of telehealth into practice., Methods: A cross-sectional 24-item web-based survey was emailed to 491 providers with ≥50 video visits (VVs) within an academic health system between 1 March 2020 and 31 December 2020. We quantitatively summarised the characteristics and perceptions of respondents by using descriptive and test statistics. We used systematic content analysis to qualitatively code open-ended responses, double coding at least 25%., Results: 247 providers (50.3%) responded to the survey. Seventy-nine per cent were confident in their ability to deliver excellent clinical care through VV. In comparison, 48% were confident in their ability to troubleshoot technical issues. Most clinicians (87%) expressed various concerns about VV. Providers across specialties generally agreed that VV reduced infection risk (71%) and transportation barriers (71%). Three overarching themes in the qualitative data included infrastructure and training, usefulness and expectation setting for patients and providers., Discussion: As healthcare systems plan for future delivery directions, they must address the tension between patients' and providers' expectations of care within the digital space. Telehealth creates new friction, one where the healthcare system must fit into the patient's life rather than the usual dynamic of the patient fitting into the healthcare system., Conclusion: Telehealth infrastructure and patient and clinician technological acumen continue to evolve. Clinicians in this survey offered valuable insights into the directions healthcare organisations can take to right-size this healthcare delivery modality., Competing Interests: Competing interests: None declared., (© Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.)