Search

BackgroundDespite the importance of informed consent in healthcare, the readability and specificity of consent forms often impedes patients’ comprehension. Health literacy is linked to patient outcomes, making it essential to address these issues. This study investigates the use of GPT-4 to simplify surgical consent forms and introduces an AI-human expert collaborative approach to validate content appropriateness.MethodsConsent forms from multiple institutions were assessed for readability and simplified using GPT-4, with pre- and post-simplification readability metrics compared using nonparametric tests. Independent reviews by medical authors and a malpractice defense attorney were conducted. Finally, GPT-4’s potential for generatingde novoprocedure-specific consent forms was assessed, with forms evaluated using a validated 8-item rubric and expert subspecialty surgeon review.ResultsAnalysis of 15 academic medical centers’ consent forms revealed significant reductions in average reading time, word rarity, and passive sentence frequency (allPP=0.004), matching the average American’s reading level. Medical and legal sufficiency consistency was confirmed. GPT-4 generated procedure-specific consent forms for five varied surgical procedures at an average 6th-grade reading level. These forms received perfect scores on a standardized consent form rubric and withstood scrutiny upon expert subspeciality surgeon review.ConclusionsThis study demonstrates the first AI-human expert collaboration to enhance surgical consent forms, significantly improving readability without sacrificing clinical detail. Our framework could be extended to other patient communication materials, emphasizing clear communication and mitigating disparities related to health literacy barriers. Ensuring AI technologies are safely incorporated into clinical practice is crucial to reach a wide range of patients, including the most vulnerable.Data Availability StatementThe raw data was not publicly deposited due to incorporating proprietary datasets, such as the Corpus of Contemporary American English. However, data are available on request for replicability.Code Availability StatementThe code used for these analyses are available from the authors on request.

BackgroundGeneral large language models (LLMs), such as ChatGPT (GPT-3.5), have demonstrated capability to pass multiple-choice medical board examinations. However, comparative accuracy of different LLMs and LLM performance on assessments of predominantly higher-order management questions is poorly understood.ObjectiveTo assess performance of three LLMs (GPT-3.5, GPT-4, and Google Bard) on a question bank designed specifically for neurosurgery oral boards examination preparation.MethodsThe 149-question Self-Assessment Neurosurgery Exam (SANS) Indications Exam was used to query LLM accuracy. Questions were input in a single best answer, multiple-choice format. Chi-squared, Fisher’s exact, and univariable logistic regression tests assessed differences in performance by question characteristics.ResultsOn a question bank with predominantly higher-order questions (85.2%), ChatGPT (GPT-3.5) and GPT-4 answered 62.4% (95% confidence interval [CI]: 54.1-70.1%) and 82.6% (95% CI: 75.2-88.1%) of questions correctly, respectively. In contrast, Bard scored 44.2% (66/149, 95% CI: 36.2-52.6%). GPT-3.5 and GPT-4 demonstrated significantly higher scores than Bard (bothPP=0.023). Among six subspecialties, GPT-4 had significantly higher accuracy in the Spine category relative to GPT-3.5 and in four categories relative to Bard (allPP=0.042) and Bard (OR=0.76,P=0.014), but not GPT-4 (OR=0.86,P=0.085). GPT-4’s performance on imaging-related questions surpassed GPT-3.5’s (68.6% vs. 47.1%,P=0.044) and was comparable to Bard’s (68.6% vs. 66.7%,P=1.000). However, GPT-4 demonstrated significantly lower rates of “hallucination” on imaging-related questions than both GPT-3.5 (2.3% vs. 57.1%,PP=0.002). Lack of question text description for imaging predicted significantly higher odds of hallucination for GPT-3.5 (OR=1.45,P=0.012) and Bard (OR=2.09,PConclusionOn a question bank of predominantly higher-order management case scenarios intended for neurosurgery oral boards preparation, GPT-4 achieved a score of 82.6%, outperforming ChatGPT and Google’s Bard.

BackgroundInterest surrounding generative large language models (LLMs) has rapidly grown. While ChatGPT (GPT-3.5), a general LLM, has shown near-passing performance on medical student board examinations, the performance of ChatGPT or its successor GPT-4 on specialized exams and the factors affecting accuracy remain unclear.ObjectiveTo assess the performance of ChatGPT and GPT-4 on a 500-question mock neurosurgical written boards examination.MethodsThe Self-Assessment Neurosurgery Exams (SANS) American Board of Neurological Surgery (ABNS) Self-Assessment Exam 1 was used to evaluate ChatGPT and GPT-4. Questions were in single best answer, multiple-choice format. Chi-squared, Fisher’s exact, and univariable logistic regression tests were employed to assess performance differences in relation to question characteristics.ResultsChatGPT (GPT-3.5) and GPT-4 achieved scores of 73.4% (95% confidence interval [CI]: 69.3-77.2%) and 83.4% (95% CI: 79.8-86.5%), respectively, relative to the user average of 73.7% (95% CI: 69.6-77.5%). Question bank users and both LLMs exceeded last year’s passing threshold of 69%. While scores between ChatGPT and question bank users were equivalent (P=0.963), GPT-4 outperformed both (bothPP=0.009) were associated with lower accuracy for ChatGPT, but not for GPT-4 (bothP>0.005). Multimodal input was not available at the time of this study so, on questions with image content, ChatGPT and GPT-4 answered 49.5% and 56.8% of questions correctly based upon contextual context clues alone.ConclusionLLMs achieved passing scores on a mock 500-question neurosurgical written board examination, with GPT-4 significantly outperforming ChatGPT.

Parkinson’s Disease (PD) is characterized by distinct motor phenomena that are expressed asynchronously. Understanding the neurophysiological correlates of these different motor states could facilitate monitoring of disease progression and allow improved assessments of therapeutic efficacy, as well as enable optimal closed-loop neuromodulation. We examined neural activity in the basal ganglia and cortex of subjects with PD during a quantitative motor task to decode tremor and bradykinesia — two cardinal motor signs of this disease — and relatively asymptomatic periods of behavior. Analysis of subcortical and cortical signals revealed that tremor and bradykinesia had distinct, nearly opposite neural signatures, while effective motor control displayed unique, differentiating features. The neurophysiological signatures of these motor states depended on the type of signal recorded as well as the location; cortical decoding accuracy generally outperformed subcortical decoding, while tremor and bradykinesia were better decoded from different portions of the subthalamic nucleus (STN). These results provide a roadmap to leverage real-time neurophysiology to understand and treat PD.One Sentence SummaryMotor signs of Parkinson’s disease such as tremor and bradykinesia can be independently decoded from subthalamic and cortical recordings.

Tremor, a common and often primary symptom of Parkinson’s disease, has been modeled with distinct onset and maintenance dynamics. To identify the neurophysiologic correlates of each state, we acquired intraoperative cortical and subthalamic nucleus recordings from ten patients performing a naturalistic visual-motor task. From this task we isolated short epochs of tremor onset and sustained tremor. Comparing these epochs, we found that the subthalamic nucleus was central to tremor onset, as it drove both motor cortical activity and tremor output. Once tremor became sustained, control of tremor shifted to cortex. At the same time, changes in directed functional connectivity across sensorimotor cortex further distinguished the sustained tremor state.SIGNIFICANCE STATEMENTTremor is a common symptom of Parkinson’s disease (PD). While tremor pathophysiology is thought to involve both basal ganglia and cerebello-thalamic-cortical circuits, it is unknown how these structures functionally interact to produce tremor. In this manuscript, we analyzed intracranial recordings from the subthalamic nucleus and sensorimotor cortex in patients with PD undergoing deep brain stimulation (DBS) surgery. Using an intraoperative task, we examined tremor in two separate dynamic contexts: when tremor first emerged, and when tremor was sustained. We believe that these findings reconcile several models of Parkinson’s tremor, while describing the short-timescale dynamics of subcortical-cortical interactions during tremor for the first time. These findings may describe a framework for developing proactive and responsive neurostimulation models for specifically treating tremor.

Objective. Identifying neural activity biomarkers of brain disease is essential to provide objective estimates of disease burden, obtain reliable feedback regarding therapeutic efficacy, and potentially to serve as a source of control for closed-loop neuromodulation. In Parkinson’s disease (PD), microelectrode recordings (MER) are routinely performed in the basal ganglia to guide electrode implantation for deep brain stimulation (DBS). While pathologically-excessive oscillatory activity has been observed and linked to PD motor dysfunction broadly, the extent to which these signals provide quantitative information about disease expression and fluctuations, particularly at short timescales, is unknown. Furthermore, the degree to which informative signal features are similar or different across patients has not been rigorously investigated. We sought to determine the extent to which motor error in PD across patients can be decoded on a rapid timescale using spectral features of neural activity. Approach. Here, we recorded neural activity from the subthalamic nucleus (STN) of subjects with PD undergoing awake DBS surgery while they performed an objective, continuous behavioral assessment that synthesized heterogenous PD motor manifestations to generate a scalar measure of motor dysfunction at short timescales. We then leveraged natural motor performance variations as a ‘ground truth’ to identify corresponding neurophysiological biomarkers. Main results. Support vector machines using multi-spectral decoding of neural signals from the STN succeeded in tracking the degree of motor impairment at short timescales (as short as one second). Spectral power across a wide range of frequencies, beyond the classic ‘β’ oscillations, contributed to this decoding, and multi-spectral models consistently outperformed those generated using more isolated frequency bands. While generalized decoding models derived across subjects were able to estimate motor impairment, patient-specific models typically performed better. Significance. These results demonstrate that quantitative information about short-timescale PD motor dysfunction is available in STN neural activity, distributed across various patient-specific spectral components, such that an individualized approach will be critical to fully harness this information for optimal disease tracking and closed-loop neuromodulation.