Search

Over 85 million computed tomography (CT) scans are performed annually in the US, of which approximately one quarter focus on the abdomen. Given the current shortage of both general and specialized radiologists, there is a large impetus to use artificial intelligence to alleviate the burden of interpreting these complex imaging studies while simultaneously using the images to extract novel physiological insights. Prior state-of-the-art approaches for automated medical image interpretation leverage vision language models (VLMs) that utilize both the image and the corresponding textual radiology reports. However, current medical VLMs are generally limited to 2D images and short reports. To overcome these shortcomings for abdominal CT interpretation, we introduce Merlin - a 3D VLM that leverages both structured electronic health records (EHR) and unstructured radiology reports for pretraining without requiring additional manual annotations. We train Merlin using a high-quality clinical dataset of paired CT scans (6+ million images from 15,331 CTs), EHR diagnosis codes (1.8+ million codes), and radiology reports (6+ million tokens) for training. We comprehensively evaluate Merlin on 6 task types and 752 individual tasks. The non-adapted (off-the-shelf) tasks include zero-shot findings classification (31 findings), phenotype classification (692 phenotypes), and zero-shot cross-modal retrieval (image to findings and image to impressions), while model adapted tasks include 5-year chronic disease prediction (6 diseases), radiology report generation, and 3D semantic segmentation (20 organs). We perform internal validation on a test set of 5,137 CTs, and external validation on 7,000 clinical CTs and on two public CT datasets (VerSe, TotalSegmentator). Beyond these clinically-relevant evaluations, we assess the efficacy of various network architectures and training strategies to depict that Merlin has favorable performance to existing task-specific baselines. We derive data scaling laws to empirically assess training data needs for requisite downstream task performance. Furthermore, unlike conventional VLMs that require hundreds of GPUs for training, we perform all training on a single GPU. This computationally efficient design can help democratize foundation model training, especially for health systems with compute constraints. We plan to release our trained models, code, and dataset, pending manual removal of all protected health information.

Cryogenic electron tomography (cryoET) is a powerful tool in structural biology, enabling detailed 3D imaging of biological specimens at a resolution of nanometers. Despite its potential, cryoET faces challenges such as the missing wedge problem, which limits reconstruction quality due to incomplete data collection angles. Recently, supervised deep learning methods leveraging convolutional neural networks (CNNs) have considerably addressed this issue; however, their pretraining requirements render them susceptible to inaccuracies and artifacts, particularly when representative training data is scarce. To overcome these limitations, we introduce a proof-of-concept unsupervised learning approach using coordinate networks (CNs) that optimizes network weights directly against input projections. This eliminates the need for pretraining, reducing reconstruction runtime by 3-20× compared to supervised methods. Our in silico results show improved shape completion and reduction of missing wedge artifacts, assessed through several voxel-based image quality metrics in real space and a novel directional Fourier Shell Correlation (FSC) metric. Our study illuminates benefits and considerations of both supervised and unsupervised approaches, guiding the development of improved reconstruction strategies.

Introduction: Epidemiological transition to NCDs is a challenge for fragile health systems in the Caribbean. The Congregations Taking Action against NCDs (CONTACT) Study intervention proposes that trained health advocates (HAs) from places of worship (PoWs), supervised by nurses at nearby primary healthcare centres (PHCs), could facilitate access to primary care among vulnerable communities. Drawing on participatory and systems thinking, we explored the capacity of local PHCs in three Caribbean countries to support this intervention., Methods: Communities in Jamaica (rural, urban), Guyana (rural) and Dominica (Indigenous Kalinago Territory) were selected for CONTACT because of their differing socio-economic, cultural, religious and health system contexts. Through mixed-method concept mapping, we co-developed a list of perceived actionable priorities (possible intervention points ranked highly for feasibility and importance) with 48 policy actors, healthcare practitioners and civic society representatives. Guided in part by the concept mapping findings, we assessed the readiness of 12 purposefully selected PHCs for the intervention, using a staff questionnaire and an observation checklist to identify enablers and constrainers., Results: Concept mapping illustrated stakeholder optimism for the intervention, but revealed perceptions of inadequate primary healthcare service capacity, resources and staff training to support implementation. Readiness assessments of PHCs identified potential enablers and constrainers that were consistent with concept mapping results. Staff support was evident. Constraints included under-staffing, which could hinder supervision of HAs; and inadequate essential NCD medicines, training in NCDs and financial and policy support for embedding community interventions. Despite a history of socio-political disadvantage, the most enabling context was found in the Kalinago Territory, where ongoing community engagement activities could support joint development of programmes between churches and PHCs., Conclusion: Multi-sectoral stakeholder consultation and direct PHC assessments revealed viability of the proposed POW-PHC partnership for NCD prevention and control. However, structural and policy support will be key for implementing change., Competing Interests: The authors have declared that no competing interests exist., (Copyright: © 2024 Gobin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.)

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care., (© 2024. The Author(s), under exclusive licence to Springer Nature America, Inc.)