Search

The foveal avascular zone (FAZ) is a retinal area devoid of capillaries and associated with multiple retinal pathologies and visual acuity. Optical Coherence Tomography Angiography (OCT-A) is a very effective means of visualizing retinal vascular and avascular areas, but its use remains limited to research settings due to its complex optics limiting availability. On the other hand, fundus photography is widely available and often adopted in population studies. In this work, we test the feasibility of estimating the FAZ from fundus photos using three different approaches. The first two approaches rely on pixel-level and image-level FAZ information to segment FAZ pixels and regress FAZ area, respectively. The third is a training mask-free pipeline combining saliency maps with an active contours approach to segment FAZ pixels while being trained on image-level measures of the FAZ areas. This enables training FAZ segmentation methods without manual alignment of fundus and OCT-A images, a time-consuming process, which limits the dataset that can be used for training. Segmentation methods trained on pixel-level labels and image-level labels had good agreement with masks from a human grader (respectively DICE of 0.45 and 0.4). Results indicate the feasibility of using fundus images as a proxy to estimate the FAZ when angiography data is not available.

Idiopathic Rem sleep Behavior Disorder (iRBD) is a significant biomarker for the development of alpha-synucleinopathies, such as Parkinson's disease (PD) or Dementia with Lewy bodies (DLB). Methods to identify patterns in iRBD patients can help in the prediction of the future conversion to these diseases during the long prodromal phase when symptoms are non-specific. These methods are essential for disease management and clinical trial recruitment. Brain PET scans with 18F-FDG PET radiotracers have recently shown promise, however, the scarcity of longitudinal data and PD/DLB conversion information makes the use of representation learning approaches such as deep convolutional networks not feasible if trained in a supervised manner. In this work, we propose a self-supervised learning strategy to learn features by comparing the brain hemispheres of iRBD non-convertor subjects, which allows for pre-training a convolutional network on a small data regimen. We introduce a loss function called hemisphere dissimilarity loss (HDL), which extends the Barlow Twins loss, that promotes the creation of invariant and non-redundant features for brain hemispheres of the same subject, and the opposite for hemispheres of different subjects. This loss enables the pre-training of a network without any information about the disease, which is then used to generate full brain feature vectors that are fine-tuned to two downstream tasks: follow-up conversion, and the type of conversion (PD or DLB) using baseline 18F-FDG PET. In our results, we find that the HDL outperforms the variational autoencoder with different forms of inputs.

Scarcity of labels for medical images is a significant barrier for training representation learning approaches based on deep neural networks. This limitation is also present when using imaging data collected during routine clinical care stored in picture archiving communication systems (PACS), as these data rarely have attached the high-quality labels required for medical image computing tasks. However, medical images extracted from PACS are commonly coupled with descriptive radiology reports that contain significant information and could be leveraged to pre-train imaging models, which could serve as starting points for further task-specific fine-tuning. In this work, we perform a head-to-head comparison of three different self-supervised strategies to pre-train the same imaging model on 3D brain computed tomography angiogram (CTA) images, with large vessel occlusion (LVO) detection as the downstream task. These strategies evaluate two natural language processing (NLP) approaches, one to extract 100 explicit radiology concepts (Rad-SpatialNet) and the other to create general-purpose radiology reports embeddings (DistilBERT). In addition, we experiment with learning radiology concepts directly or by using a recent self-supervised learning approach (CLIP) that learns by ranking the distance between language and image vector embeddings. The LVO detection task was selected because it requires 3D imaging data, is clinically important, and requires the algorithm to learn outputs not explicitly stated in the radiology report. Pre-training was performed on an unlabeled dataset containing 1,542 3D CTA - reports pairs. The downstream task was tested on a labeled dataset of 402 subjects for LVO. We find that the pre-training performed with CLIP-based strategies improve the performance of the imaging model to detect LVO compared to a model trained only on the labeled data. The best performance was achieved by pre-training using the explicit radiology concepts and CLIP strategy.