Search

Optical coherence tomography (OCT) has become critical for diagnosing retinal diseases as it enables 3D images of the retina and optic nerve. OCT acquisition is fast, non-invasive, affordable, and scalable. Due to its broad applicability, massive numbers of OCT images have been accumulated in routine exams, making it possible to train large-scale foundation models that can generalize to various diagnostic tasks using OCT images. Nevertheless, existing foundation models for OCT only consider 2D image slices, overlooking the rich 3D structure. Here, we present OCTCube, a 3D foundation model pre-trained on 26,605 3D OCT volumes encompassing 1.62 million 2D OCT images. OCTCube is developed based on 3D masked autoencoders and exploits FlashAttention to reduce the larger GPU memory usage caused by modeling 3D volumes. OCTCube outperforms 2D models when predicting 8 retinal diseases in both inductive and cross-dataset settings, indicating that utilizing the 3D structure in the model instead of 2D data results in significant improvement. OCTCube further shows superior performance on cross-device prediction and when predicting systemic diseases, such as diabetes and hypertension, further demonstrating its strong generalizability. Finally, we propose a contrastive-self-supervised-learning-based OCT-IR pre-training framework (COIP) for cross-modality analysis on OCT and infrared retinal (IR) images, where the OCT volumes are embedded using OCTCube. We demonstrate that COIP enables accurate alignment between OCT and IR en face images. Collectively, OCTCube, a 3D OCT foundation model, demonstrates significantly better performance against 2D models on 27 out of 29 tasks and comparable performance on the other two tasks, paving the way for AI-based retinal disease diagnosis.

One of the most poorly understood aspects of low-mass star formation is how multiple-star systems are formed. Here we present the results of Atacama Large Millimeter/submillimeter Array (ALMA) Band-6 observations towards a forming quadruple protostellar system, G206.93-16.61E2, in the Orion B molecular cloud. ALMA 1.3 mm continuum emission reveals four compact objects, of which two are Class I young stellar objects (YSOs), and the other two are likely in prestellar phase. The 1.3 mm continuum emission also shows three asymmetric ribbon-like structures that are connected to the four objects, with lengths ranging from $\sim$500 au to $\sim$2200 au. By comparing our data with magneto-hydrodynamic (MHD) simulations, we suggest that these ribbons trace accretion flows and also function as gas bridges connecting the member protostars. Additionally, ALMA CO J=2-1 line emission reveals a complicated molecular outflow associated with G206.93-16.61E2 with arc-like structures suggestive of an outflow cavity viewed pole-on., Comment: The paper was accepted by APJL

Better understanding of Large Language Models' (LLMs) legal analysis abilities can contribute to improving the efficiency of legal services, governing artificial intelligence, and leveraging LLMs to identify inconsistencies in law. This paper explores LLM capabilities in applying tax law. We choose this area of law because it has a structure that allows us to set up automated validation pipelines across thousands of examples, requires logical reasoning and maths skills, and enables us to test LLM capabilities in a manner relevant to real-world economic lives of citizens and companies. Our experiments demonstrate emerging legal understanding capabilities, with improved performance in each subsequent OpenAI model release. We experiment with retrieving and utilising the relevant legal authority to assess the impact of providing additional legal context to LLMs. Few-shot prompting, presenting examples of question-answer pairs, is also found to significantly enhance the performance of the most advanced model, GPT-4. The findings indicate that LLMs, particularly when combined with prompting enhancements and the correct legal texts, can perform at high levels of accuracy but not yet at expert tax lawyer levels. As LLMs continue to advance, their ability to reason about law autonomously could have significant implications for the legal profession and AI governance.

Purpose - To develop and validate a deep learning (DL) framework for the detection and quantification of drusen and reticular pseudodrusen (RPD) on optical coherence tomography scans. Design - Development and validation of deep learning models for classification and feature segmentation. Methods - A DL framework was developed consisting of a classification model and an out-of-distribution (OOD) detection model for the identification of ungradable scans; a classification model to identify scans with drusen or RPD; and an image segmentation model to independently segment lesions as RPD or drusen. Data were obtained from 1284 participants in the UK Biobank (UKBB) with a self-reported diagnosis of age-related macular degeneration (AMD) and 250 UKBB controls. Drusen and RPD were manually delineated by five retina specialists. The main outcome measures were sensitivity, specificity, area under the ROC curve (AUC), kappa, accuracy and intraclass correlation coefficient (ICC). Results - The classification models performed strongly at their respective tasks (0.95, 0.93, and 0.99 AUC, respectively, for the ungradable scans classifier, the OOD model, and the drusen and RPD classification model). The mean ICC for drusen and RPD area vs. graders was 0.74 and 0.61, respectively, compared with 0.69 and 0.68 for intergrader agreement. FROC curves showed that the model's sensitivity was close to human performance. Conclusions - The models achieved high classification and segmentation performance, similar to human performance. Application of this robust framework will further our understanding of RPD as a separate entity from drusen in both research and clinical settings., Comment: 26 pages, 7 figures

Beginning with cosmological initial conditions at z=100, we simulate the effects of magnetic fields on the formation of Population III stars and compare our results with the predictions of Paper I. We use Gadget-2 to follow the evolution of the system while the field is weak. We introduce a new method for treating kinematic fields by tracking the evolution of the deformation tensor. The growth rate in this stage of the simulation is lower than expected for diffuse astrophysical plasmas, which have a very low resistivity (high magnetic Prandtl number); we attribute this to the large numerical resistivity in simulations, corresponding to a magnetic Prandtl number of order unity. When the magnetic field begins to be dynamically significant in the core of the minihalo at z=27, we map it onto a uniform grid and follow the evolution in an adaptive mesh refinement, MHD simulation in Orion2. The nonlinear evolution of the field in the Orion2 simulation violates flux-freezing and is consistent with the theory proposed by Xu & Lazarian. The fields approach equipartition with kinetic energy at densities ~ 10^10 - 10^12 cm^-3. When the same calculation is carried out in Orion2 with no magnetic fields, several protostars form, ranging in mass from ~ 1 to 30 M_sol with magnetic fields, only a single ~ 30 M_sol protostar forms by the end of the simulation. Magnetic fields thus suppress the formation of low-mass Pop III stars, yielding a top-heavy Pop III IMF and contributing to the absence of observed Pop III stars., Comment: 31 pages, 30 figures, accepted to MNRAS

Unsupervised cross domain adaptation for OCT 7 layer segmentation and other medical applications where labeled training data is only available in a source domain and unavailable in the target domain. Our proposed method helps generalize of deep learning to many areas in the medical field where labeled training data are expensive and time consuming to acquire or where target domains are too novel to have had labelling.

A common problem in machine learning is determining if a variable significantly contributes to a model's prediction performance. This problem is aggravated for datasets, such as gene expression datasets, that suffer the worst case of dimensionality: a low number of observations along with a high number of possible explanatory variables. In such scenarios, traditional methods for testing variable statistical significance or constructing variable confidence intervals do not apply. To address these problems, we developed a novel permutation framework for testing the significance of variables in supervised models. Our permutation framework has three main advantages. First, it is non-parametric and does not rely on distributional assumptions or asymptotic results. Second, it not only ranks model variables in terms of relative importance, but also tests for statistical significance of each variable. Third, it can test for the significance of the interaction between model variables. We applied this permutation framework to multi-class classification of the Iris flower dataset and of brain regions in RNA expression data, and using this framework showed variable-level statistical significance and interactions.

We investigate the time evolution of dense cores identified in molecular cloud simulations using dendrograms, which are a common tool to identify hierarchical structure in simulations and observations of star formation. We develop an algorithm to link dendrogram structures through time using the three-dimensional density field from magnetohydrodynamical simulations, thus creating histories for all dense cores in the domain. We find that the population-wide distributions of core properties are relatively invariant in time, and quantities like the core mass function match with observations. Despite this consistency, an individual core may undergo large (>40%), stochastic variations due to the redefinition of the dendrogram structure between timesteps. This variation occurs independent of environment and stellar content. We identify a population of short-lived (<200 kyr) overdensities masquerading as dense cores that may comprise ~20% of any time snapshot. Finally, we note the importance of considering the full history of cores when interpreting the origin of the initial mass function; we find that, especially for systems containing multiple stars, the core mass defined by a dendrogram leaf in a snapshot is typically less than the final system stellar mass. This work reinforces that there is no time-stable density contour that defines a star-forming core. The dendrogram itself can induce significant structure variation between timesteps due to small changes in the density field. Thus, one must use caution when comparing dendrograms of regions with different ages or environment properties because differences in dendrogram structure may not come solely from the physical evolution of dense cores., Comment: 20 pages, 17 figures. Submitted to MNRAS

Stars rarely form in isolation. Nearly half of the stars in the Milky Way have a companion, and this fraction increases in star-forming regions. However, why some dense cores and filaments form bound pairs while others form single stars remains unclear. We present a set of three-dimensional, gravo-magnetohydrodynamic simulations of turbulent star-forming clouds, aimed at understanding the formation and evolution of multiple-star systems formed through large scale (>~$10^3$ AU) turbulent fragmentation. We investigate three global magnetic field strengths, with global mass-to-flux ratios of $\mu_\phi$=2, 8, and 32. The initial separations of protostars in multiples depends on the global magnetic field strength, with stronger magnetic fields (e.g., $\mu_\phi$=2) suppressing fragmentation on smaller scales. The overall multiplicity fraction (MF) is between 0.4-0.6 for our strong and intermediate magnetic field strengths, which is in agreement with observations. The weak field case has a lower fraction. The MF is relatively constant throughout the simulations, even though stellar densities increase as collapse continues. While the MF rarely exceeds 60% in all three simulations, over 80% of all protostars are part of a binary system at some point. We additionally find that the distribution of binary spin mis-alignment angles is consistent with a randomized distribution. In all three simulations, several binaries originate with wide separations and dynamically evolve to <~ $10^2$ AU separations. We show that a simple model of mass accretion and dynamical friction with the gas can explain this orbital evolution., Comment: Accepted to ApJ. 28 pages, 23 figures, comments from the community welcomed

Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g. same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strategies to accommodate common scenarios in multi-site, multi-scanner clinical imaging data sets. We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy. Specifically, the method was trained on a large set of 3,975 subjects from the UK Biobank. It was then directly tested on 600 different subjects from the UK Biobank for intra-domain testing and two other sets for cross-domain testing: the ACDC dataset (100 subjects, 1 site, 2 scanners) and the BSCMR-AS dataset (599 subjects, 6 sites, 9 scanners). The proposed method produces promising segmentation results on the UK Biobank test set which are comparable to previously reported values in the literature, while also performing well on cross-domain test sets, achieving a mean Dice metric of 0.90 for the left ventricle, 0.81 for the myocardium and 0.82 for the right ventricle on the ACDC dataset; and 0.89 for the left ventricle, 0.83 for the myocardium on the BSCMR-AS dataset. The proposed method offers a potential solution to improve CNN-based model generalizability for the cross-scanner and cross-site cardiac MR image segmentation task., Comment: 15 pages, 8 figures

Background: The trend towards large-scale studies including population imaging poses new challenges in terms of quality control (QC). This is a particular issue when automatic processing tools, e.g. image segmentation methods, are employed to derive quantitative measures or biomarkers for later analyses. Manual inspection and visual QC of each segmentation isn't feasible at large scale. However, it's important to be able to automatically detect when a segmentation method fails so as to avoid inclusion of wrong measurements into subsequent analyses which could lead to incorrect conclusions. Methods: To overcome this challenge, we explore an approach for predicting segmentation quality based on Reverse Classification Accuracy, which enables us to discriminate between successful and failed segmentations on a per-cases basis. We validate this approach on a new, large-scale manually-annotated set of 4,800 cardiac magnetic resonance scans. We then apply our method to a large cohort of 7,250 cardiac MRI on which we have performed manual QC. Results: We report results used for predicting segmentation quality metrics including Dice Similarity Coefficient (DSC) and surface-distance measures. As initial validation, we present data for 400 scans demonstrating 99% accuracy for classifying low and high quality segmentations using predicted DSC scores. As further validation we show high correlation between real and predicted scores and 95% classification accuracy on 4,800 scans for which manual segmentations were available. We mimic real-world application of the method on 7,250 cardiac MRI where we show good agreement between predicted quality metrics and manual visual QC scores. Conclusions: We show that RCA has the potential for accurate and fully automatic segmentation QC on a per-case basis in the context of large-scale population imaging as in the UK Biobank Imaging Study., Comment: 14 pages, 7 figures, Journal of Cardiovascular Magnetic Resonance

Recent advances in deep learning based image segmentation methods have enabled real-time performance with human-level accuracy. However, occasionally even the best method fails due to low image quality, artifacts or unexpected behaviour of black box algorithms. Being able to predict segmentation quality in the absence of ground truth is of paramount importance in clinical practice, but also in large-scale studies to avoid the inclusion of invalid data in subsequent analysis. In this work, we propose two approaches of real-time automated quality control for cardiovascular MR segmentations using deep learning. First, we train a neural network on 12,880 samples to predict Dice Similarity Coefficients (DSC) on a per-case basis. We report a mean average error (MAE) of 0.03 on 1,610 test samples and 97% binary classification accuracy for separating low and high quality segmentations. Secondly, in the scenario where no manually annotated data is available, we train a network to predict DSC scores from estimated quality obtained via a reverse testing strategy. We report an MAE=0.14 and 91% binary classification accuracy for this case. Predictions are obtained in real-time which, when combined with real-time segmentation methods, enables instant feedback on whether an acquired scan is analysable while the patient is still in the scanner. This further enables new applications of optimising image acquisition towards best possible analysis results., Comment: Accepted at MICCAI 2018

Purpose: To determine if deep learning networks could be trained to forecast a future 24-2 Humphrey Visual Field (HVF). Participants: All patients who obtained a HVF 24-2 at the University of Washington. Methods: All datapoints from consecutive 24-2 HVFs from 1998 to 2018 were extracted from a University of Washington database. Ten-fold cross validation with a held out test set was used to develop the three main phases of model development: model architecture selection, dataset combination selection, and time-interval model training with transfer learning, to train a deep learning artificial neural network capable of generating a point-wise visual field prediction. Results: More than 1.7 million perimetry points were extracted to the hundredth decibel from 32,443 24-2 HVFs. The best performing model with 20 million trainable parameters, CascadeNet-5, was selected. The overall MAE for the test set was 2.47 dB (95% CI: 2.45 dB to 2.48 dB). The 100 fully trained models were able to successfully predict progressive field loss in glaucomatous eyes up to 5.5 years in the future with a correlation of 0.92 between the MD of predicted and actual future HVF (p < 2.2 x 10 -16 ) and an average difference of 0.41 dB. Conclusions: Using unfiltered real-world datasets, deep learning networks show an impressive ability to not only learn spatio-temporal HVF changes but also to generate predictions for future HVFs up to 5.5 years, given only a single HVF.

Despite significant advances in artificial intelligence (AI) for computer vision, its application in medical imaging has been limited by the burden and limits of expert-generated labels. We used images from optical coherence tomography angiography (OCTA), a relatively new imaging modality that measures perfusion of the retinal vasculature, to train an AI algorithm to generate vasculature maps from standard structural optical coherence tomography (OCT) images of the same retinae, both exceeding the ability and bypassing the need for expert labeling. Deep learning was able to infer perfusion of microvasculature from structural OCT images with similar fidelity to OCTA and significantly better than expert clinicians (P < 0.00001). OCTA suffers from need of specialized hardware, laborious acquisition protocols, and motion artifacts; whereas our model works directly from standard OCT which are ubiquitous and quick to obtain, and allows unlocking of large volumes of previously collected standard OCT data both in existing clinical trials and clinical practice. This finding demonstrates a novel application of AI to medical imaging, whereby subtle regularities between different modalities are used to image the same body part and AI is used to generate detailed and accurate inferences of tissue function from structure imaging., Comment: Under revision at Nature Communications. Submitted on June 5th 2017

Cardiovascular magnetic resonance (CMR) imaging is a standard imaging modality for assessing cardiovascular diseases (CVDs), the leading cause of death globally. CMR enables accurate quantification of the cardiac chamber volume, ejection fraction and myocardial mass, providing information for diagnosis and monitoring of CVDs. However, for years, clinicians have been relying on manual approaches for CMR image analysis, which is time consuming and prone to subjective errors. It is a major clinical challenge to automatically derive quantitative and clinically relevant information from CMR images. Deep neural networks have shown a great potential in image pattern recognition and segmentation for a variety of tasks. Here we demonstrate an automated analysis method for CMR images, which is based on a fully convolutional network (FCN). The network is trained and evaluated on a large-scale dataset from the UK Biobank, consisting of 4,875 subjects with 93,500 pixelwise annotated images. The performance of the method has been evaluated using a number of technical metrics, including the Dice metric, mean contour distance and Hausdorff distance, as well as clinically relevant measures, including left ventricle (LV) end-diastolic volume (LVEDV) and end-systolic volume (LVESV), LV mass (LVM); right ventricle (RV) end-diastolic volume (RVEDV) and end-systolic volume (RVESV). By combining FCN with a large-scale annotated dataset, the proposed automated method achieves a high performance on par with human experts in segmenting the LV and RV on short-axis CMR images and the left atrium (LA) and right atrium (RA) on long-axis CMR images., Comment: Accepted for publication by Journal of Cardiovascular Magnetic Resonance