Search

Abstract Large language models (LLMs) hold great promise in summarizing medical evidence. Most recent studies focus on the application of proprietary LLMs. Using proprietary LLMs introduces multiple risk factors, including a lack of transparency and vendor dependency. While open-source LLMs allow better transparency and customization, their performance falls short compared to the proprietary ones. In this study, we investigated to what extent fine-tuning open-source LLMs can further improve their performance. Utilizing a benchmark dataset, MedReview, consisting of 8161 pairs of systematic reviews and summaries, we fine-tuned three broadly-used, open-sourced LLMs, namely PRIMERA, LongT5, and Llama-2. Overall, the performance of open-source models was all improved after fine-tuning. The performance of fine-tuned LongT5 is close to GPT-3.5 with zero-shot settings. Furthermore, smaller fine-tuned models sometimes even demonstrated superior performance compared to larger zero-shot models. The above trends of improvement were manifested in both a human evaluation and a larger-scale GPT4-simulated evaluation.

Background: Recognizing sport-related concussion (SRC) is challenging and relies heavily on subjective symptom reports. An objective, biological marker could improve recognition and understanding of SRC. There is emerging evidence that salivary micro-ribonucleic acids (miRNAs) may serve as biomarkers of concussion; however, it remains unclear whether concussion-related miRNAs are impacted by exercise. We sought to determine whether 40 miRNAs previously implicated in concussion pathophysiology were affected by participation in a variety of contact and non-contact sports. Our goal was to refine a miRNA-based tool capable of identifying athletes with SRC without the confounding effects of exercise. Methods: This case-control study harmonized data from concussed and non-concussed athletes recruited across 10 sites. Levels of salivary miRNAs within 455 samples from 314 individuals were measured with RNA sequencing. Within-subjects testing was used to identify and exclude miRNAs that changed with either (a) a single episode of exercise (166 samples from 83 individuals) or (b) season-long participation in contact sports (212 samples from 106 individuals). The miRNAs that were not impacted by exercise were interrogated for SRC diagnostic utility using logistic regression (172 samples from 75 concussed and 97 non-concussed individuals). Results: Two miRNAs (miR-532-5p and miR-182-5p) decreased (adjusted p < 0.05) after a single episode of exercise, and 1 miRNA (miR-4510) increased only after contact sports participation. Twenty-three miRNAs changed at the end of a contact sports season. Two of these miRNAs (miR-26b-3p and miR-29c-3p) were associated (R > 0.50; adjusted p < 0.05) with the number of head impacts sustained in a single football practice. Among the 15 miRNAs not confounded by exercise or season-long contact sports participation, 11 demonstrated a significant difference (adjusted p < 0.05) between concussed and non-concussed participants, and 6 displayed moderate ability (area under curve > 0.70) to identify concussion. A single ratio (miR-27a-5p/miR-30a-3p) displayed the highest accuracy (AUC = 0.810, sensitivity = 82.4%, specificity = 73.3%) for differentiating concussed and non-concussed participants. Accuracy did not differ between participants with SRC and non-SRC (z = 0.5, p = 0.60). Conclusion: Salivary miRNA levels may accurately identify SRC when not confounded by exercise. Refinement of this approach in a large cohort of athletes could eventually lead to a non-invasive, sideline adjunct for SRC assessment.

Enterprise data warehouses for research (EDW4R) is a critical component of National Institutes of Health Clinical and Translational Science Award (CTSA) hubs. EDW4R operations have unique needs that require specialized skills and collaborations across multiple domains which limit the ability to apply existing models of information technology (IT) performance. Because of this uniqueness, we developed a new EDW4R maturity model based on prior qualitative study of operational practices for supporting EDW4Rs at CTSA hubs. In a pilot study, respondents from fifteen CTSA hubs completed the novel EDW4R maturity index survey by rating 33 maturity statements across 6 categories using a 5-point Likert scale. Of the six categories, respondents rated workforce as most mature (4.17 [3.67–4.42]) and relationship with enterprise IT as the least mature (3.00 [2.80–3.80]). Our pilot of a novel maturity index shows a baseline quantitative measure of EDW4R functions across fifteen CTSA hubs. The maturity index may be useful to faculty and staff currently leading an EDW4R by creating opportunities to explore the index in local context and comparison to other institutions.

Background: In the global effort to prevent death by suicide, many academic medical institutions are implementing natural language processing (NLP) approaches to detect suicidality from unstructured clinical text in electronic health records (EHRs), with the hope of targeting timely, preventative interventions to individuals most at risk of suicide. Despite the international need, the development of these NLP approaches in EHRs has been largely local and not shared across healthcare systems. Methods: In this study, we developed a process to share NLP approaches that were individually developed at King's College London (KCL), UK and Weill Cornell Medicine (WCM), US - two academic medical centers based in different countries with vastly different healthcare systems. We tested and compared the algorithms’ performance on manually annotated clinical notes (KCL: n = 4,911 and WCM = 837). Results: After a successful technical porting of the NLP approaches, our quantitative evaluation determined that independently developed NLP approaches can detect suicidality at another healthcare organization with a different EHR system, clinical documentation processes, and culture, yet do not achieve the same level of success as at the institution where the NLP algorithm was developed (KCL approach: F1-score 0.85 vs. 0.68, WCM approach: F1-score 0.87 vs. 0.72). Limitations: Independent NLP algorithm development and patient cohort selection at the two institutions comprised direct comparability. Conclusions: Shared use of these NLP approaches is a critical step forward towards improving data-driven algorithms for early suicide risk identification and timely prevention.

Abstract COVID-19-associated respiratory failure offers the unprecedented opportunity to evaluate the differential host response to a uniform pathogenic insult. Understanding whether there are distinct subphenotypes of severe COVID-19 may offer insight into its pathophysiology. Sequential Organ Failure Assessment (SOFA) score is an objective and comprehensive measurement that measures dysfunction severity of six organ systems, i.e., cardiovascular, central nervous system, coagulation, liver, renal, and respiration. Our aim was to identify and characterize distinct subphenotypes of COVID-19 critical illness defined by the post-intubation trajectory of SOFA score. Intubated COVID-19 patients at two hospitals in New York city were leveraged as development and validation cohorts. Patients were grouped into mild, intermediate, and severe strata by their baseline post-intubation SOFA. Hierarchical agglomerative clustering was performed within each stratum to detect subphenotypes based on similarities amongst SOFA score trajectories evaluated by Dynamic Time Warping. Distinct worsening and recovering subphenotypes were identified within each stratum, which had distinct 7-day post-intubation SOFA progression trends. Patients in the worsening suphenotypes had a higher mortality than those in the recovering subphenotypes within each stratum (mild stratum, 29.7% vs. 10.3%, p = 0.033; intermediate stratum, 29.3% vs. 8.0%, p = 0.002; severe stratum, 53.7% vs. 22.2%, p

Here, using clinical samples and autopsy tissues, the authors combine fast-colorimetric test (LAMP) for SARS-CoV-2 infection and large-scale shotgun metatranscriptomics for host, viral, and microbial profiling and provide a map of the viral genetic features of the New York City outbreak and associate specific host responses and gene expression perturbations with SARS-CoV-2 infection.

Background: Existing risk assessment tools for heart failure (HF) outcomes use structured databases with static, single-timepoint clinical data and have limited accuracy. Objective: The purpose of this study was to develop a comprehensive approach for accurate prediction of 30-day unplanned readmission and all-cause mortality (ACM) that integrates clinical and physiological data available in the electronic health record system. Methods: Three predictive models for 30-day unplanned readmissions or ACM were created using an extreme gradient boosting approach: (1) index admission model; (2) index discharge model; and (3) feature-aggregated model. Performance was assessed by the area under the curve (AUC) metric and compared with that of the HOSPITAL score, a widely used predictive model for hospital readmission. Results: A total of 3774 patients with a primary billing diagnosis of HF were included (614 experienced the primary outcome), with 796 variables used in the admission and discharge models, and 2032 in the feature-aggregated model. The index admission model had AUC = 0.723, the index discharge model had AUC = 0.754, and the feature-aggregated model had AUC = 0.756 for prediction of 30-day unplanned readmission or ACM. For comparison, the HOSPITAL score had AUC = 0.666 (admission model: P = .093; discharge model: P = .022; feature aggregated: P = .012). Conclusion: These models predict risk of HF hospitalizations and ACM in patients admitted with HF and emphasize the importance of incorporating large numbers of variables in machine learning models to identify predictors for future investigation.

Abstract Objective Atrial cardiopathy without atrial fibrillation (AF) may be a potential cardiac source of embolic strokes of undetermined source (ESUS). Atrial volume is a feature of atrial cardiopathy, but the relationship between atrial volume and ESUS remains unclear. Methods We compared left atrial volume among ischemic stroke subtypes in the Cornell Acute Stroke Academic Registry (CAESAR), which includes all patients with acute ischemic stroke at our hospital since 2011. Stroke subtype was determined by neurologists per the TOAST classification and consensus ESUS definition. Left atrial volume index (LAVI) was obtained directly from our echocardiography image system (Xcelera, Philips Healthcare). We used t‐tests and analysis of variance for unadjusted comparisons and targeted minimum loss‐based estimation for comparisons adjusted for demographics and comorbidities. Results Among 2116 patients in CAESAR from 2011 to 2016, 1293 had LAVI measurements. LAVI varied across subtypes (P

Abstract Background Early, accurate diagnosis of mild traumatic brain injury (mTBI) can improve clinical outcomes for patients, but mTBI remains difficult to diagnose because of reliance on subjective symptom reports. An objective biomarker could increase diagnostic accuracy and improve clinical outcomes. The aim of this study was to assess the ability of salivary noncoding RNA (ncRNA) to serve as a diagnostic adjunct to current clinical tools. We hypothesized that saliva ncRNA levels would demonstrate comparable accuracy for identifying mTBI as measures of symptom burden, neurocognition, and balance. Methods This case‐control study involved 538 individuals. Participants included 251 individuals with mTBI, enrolled ≤14 days postinjury, from 11 clinical sites. Saliva samples (n = 679) were collected at five time points (≤3, 4‐7, 8‐14, 15‐30, and 31‐60 days post‐mTBI). Levels of ncRNAs (microRNAs, small nucleolar RNAs, and piwi‐interacting RNAs) were quantified within each sample using RNA sequencing. The first sample from each mTBI participant was compared to saliva samples from 287 controls. Samples were divided into testing (n = 430; mTBI = 201 and control = 239) and training sets (n = 108; mTBI = 50 and control = 58). The test set was used to identify ncRNA diagnostic candidates and create a diagnostic model. Model accuracy was assessed in the naïve test set. Results A model utilizing seven ncRNA ratios, along with participant age and chronic headache status, differentiated mTBI and control participants with a cross‐validated area under the curve (AUC) of .857 in the training set (95% CI, .816‐.903) and .823 in the naïve test set. In a subset of participants (n = 321; mTBI = 176 and control = 145) assessed for symptom burden (Post‐Concussion Symptom Scale), as well as neurocognition and balance (ClearEdge System), these clinical measures yielded cross‐validated AUC of .835 (95% CI, .782‐.880) and .853 (95% CI, .803‐.899), respectively. A model employing symptom burden and four neurocognitive measures identified mTBI participants with similar AUC (.888; CI, .845‐.925) as symptom burden and four ncRNAs (.932; 95% CI, .890‐.965). Conclusion Salivary ncRNA levels represent a noninvasive, biologic measure that can aid objective, accurate diagnosis of mTBI.