Search

Summary: Background: Large language models (LLMs) have shown promising performance in various healthcare domains, but their effectiveness in identifying specific clinical conditions in real medical records is less explored. This study evaluates LLMs for detecting signs of cognitive decline in real electronic health record (EHR) clinical notes, comparing their error profiles with traditional models. The insights gained will inform strategies for performance enhancement. Methods: This study, conducted at Mass General Brigham in Boston, MA, analysed clinical notes from the four years prior to a 2019 diagnosis of mild cognitive impairment in patients aged 50 and older. We developed prompts for two LLMs, Llama 2 and GPT-4, on Health Insurance Portability and Accountability Act (HIPAA)-compliant cloud-computing platforms using multiple approaches (e.g., hard prompting, retrieval augmented generation, and error analysis-based instructions) to select the optimal LLM-based method. Baseline models included a hierarchical attention-based neural network and XGBoost. Subsequently, we constructed an ensemble of the three models using a majority vote approach. Confusion-matrix-based scores were used for model evaluation. Findings: We used a randomly annotated sample of 4949 note sections from 1969 patients (women: 1046 [53.1%]; age: mean, 76.0 [SD, 13.3] years), filtered with keywords related to cognitive functions, for model development. For testing, a random annotated sample of 1996 note sections from 1161 patients (women: 619 [53.3%]; age: mean, 76.5 [SD, 10.2] years) without keyword filtering was utilised. GPT-4 demonstrated superior accuracy and efficiency compared to Llama 2, but did not outperform traditional models. The ensemble model outperformed the individual models in terms of all evaluation metrics with statistical significance (p

Background: Initial insights into oncology clinical trial outcomes are often gleaned manually from conference abstracts. We aimed to develop an automated system to extract safety and efficacy information from study abstracts with high precision and fine granularity, transforming them into computable data for timely clinical decision-making. Methods: We collected clinical trial abstracts from key conferences and PubMed (2012–2023). The SEETrials system was developed with three modules: preprocessing, prompt engineering with knowledge ingestion, and postprocessing. We evaluated the system's performance qualitatively and quantitatively and assessed its generalizability across different cancer types— multiple myeloma (MM), breast, lung, lymphoma, and leukemia. Furthermore, the efficacy and safety of innovative therapies, including CAR-T, bispecific antibodies, and antibody-drug conjugates (ADC), in MM were analyzed across a large scale of clinical trial studies. Results: SEETrials achieved high precision (0.964), recall (sensitivity) (0.988), and F1 score (0.974) across 70 data elements present in the MM trial studies Generalizability tests on four additional cancers yielded precision, recall, and F1 scores within the 0.979–0.992 range. Variation in the distribution of safety and efficacy-related entities was observed across diverse therapies, with certain adverse events more common in specific treatments. Comparative performance analysis using overall response rate (ORR) and complete response (CR) highlighted differences among therapies: CAR-T (ORR: 88 %, 95 % CI: 84–92 %; CR: 95 %, 95 % CI: 53–66 %), bispecific antibodies (ORR: 64 %, 95 % CI: 55–73 %; CR: 27 %, 95 % CI: 16–37 %), and ADC (ORR: 51 %, 95 % CI: 37–65 %; CR: 26 %, 95 % CI: 1–51 %). Notable study heterogeneity was identified (>75 % I2 heterogeneity index scores) across several outcome entities analyzed within therapy subgroups. Conclusion: SEETrials demonstrated highly accurate data extraction and versatility across different therapeutics and various cancer domains. Its automated processing of large datasets facilitates nuanced data comparisons, promoting the swift and effective dissemination of clinical insights.

The FGF family of extracellular signaling factors has been proposed to play multiple roles in patterning the telencephalon, the precursor to the cerebrum. In this study, unlike previous ones, we effectively abolish FGF signaling in the anterior neural plate via deletion of three FGF receptor (FGFR) genes. Triple FGFR mutant mice exhibit a complete loss of the telencephalon, except the dorsal midline. Disruption of FGF signaling prior to and coincident with telencephalic induction reveals that FGFs promote telencephalic character and are strictly required to keep telencephalic cells alive. Moreover, progressively more severe truncations of the telencephalon are observed in FGFR single, double and triple mutants. Together with previous gain-of-function studies showing induction of Foxg1 expression and mirror-image duplications of the cortex by exogenous FGF8, our loss-of-function results suggest that, rather than independently patterning different areas, FGF ligands and receptors act in concert to mediate organizer activity for the whole telencephalon. [ABSTRACT FROM AUTHOR]

Sonic hedgehog (SHH) is required to generate ventral cell types throughout the central nervous system. Its role in directly specifying ventral cells, however, has recently been questioned because loss of the Shh gene has little effect on ventral development if the Gli3 gene is also mutant. Consequently, another ventral determinant must exist. Here, genetic evidence establishes that FGFs are required for ventral telencephalon development. First, simultaneous deletion of Fgfr1 and Fgfr3 specifically in the telencephalon results in the loss of differentiated ventromedial cells; and second, in the Fgfr1;Fgfr2 double mutant, ventral precursor cells are lost, mimicking the phenotype obtained previously with a loss of SHH signalling. Yet, in the Fgfr1;Fgfr2 mutant, Shh remains expressed, as does Gli1, the transcription of which depends on SHH activity, suggesting that FGF signalling acts independently of SHH to generate ventral precursors. Moreover, the Fgfr1;Fgfr2 phenotype, unlike the Shh phenotype, is not rescued by loss of Gli3, further indicating that FGFs act downstream of Shh and Gli3 to generate ventral telencephalic cell types. [ABSTRACT FROM AUTHOR]