Search

Weakly-supervised learning (WSL) has been proposed to alleviate the conflict between data annotation cost and model performance through employing sparsely-grained (i.e., point-, box-, scribble-wise) supervision and has shown promising performance, particularly in the image segmentation field. However, it is still a very challenging task due to the limited supervision, especially when only a small number of labeled samples are available. Additionally, almost all existing WSL segmentation methods are designed for star-convex structures which are very different from curvilinear structures such as vessels and nerves. In this paper, we propose a novel sparsely annotated segmentation framework for curvilinear structures, named YoloCurvSeg. A very essential component of YoloCurvSeg is image synthesis. Specifically, a background generator delivers image backgrounds that closely match the real distributions through inpainting dilated skeletons. The extracted backgrounds are then combined with randomly emulated curves generated by a Space Colonization Algorithm-based foreground generator and through a multilayer patch-wise contrastive learning synthesizer. In this way, a synthetic dataset with both images and curve segmentation labels is obtained, at the cost of only one or a few noisy skeleton annotations. Finally, a segmenter is trained with the generated dataset and possibly an unlabeled dataset. The proposed YoloCurvSeg is evaluated on four publicly available datasets (OCTA500, CORN, DRIVE and CHASEDB1) and the results show that YoloCurvSeg outperforms state-of-the-art WSL segmentation methods by large margins. With only one noisy skeleton annotation (respectively 0.14%, 0.03%, 1.40%, and 0.65% of the full annotation), YoloCurvSeg achieves more than 97% of the fully-supervised performance on each dataset. Code and datasets will be released at https://github.com/llmir/YoloCurvSeg., 19 pages, 14 figures, submitted to MEDIA

Multi-modality magnetic resonance (MR) images provide complementary information for disease diagnoses. However, modality missing is quite usual in real-life clinical practice. Current methods usually employ convolution-based generative adversarial network (GAN) or its variants to synthesize the missing modality. With the development of vision transformer, we explore its application in the MRI modality synthesis task in this work. We propose a novel supervised deep learning method for synthesizing a missing modality, making use of a transformer-based encoder. Specifically, a model is trained for translating 2D MR images from T1-weighted to T2-weighted based on conditional GAN (cGAN). We replace the encoder with transformer and input adjacent slices to enrich spatial prior knowledge. Experimental results on a private dataset and a public dataset demonstrate that our proposed model outperforms state-of-the-art supervised methods for MR image synthesis, both quantitatively and qualitatively. Clinical relevance- This work proposes a method to synthesize T2-weighted images from T1-weighted ones to address the modality missing issue in MRI.

Fundus images of poor quality may seriously influence clinic judgments. Existing fundus image quality enhancement (FIQE) approaches mainly make use of general image features but no prior domain knowledge. In this paper, we proposed and validated an efficient FIQE method with a prior constraint, named Efficient Prior Contrastive unpaired Generative Adversarial Network (EPC-GAN). Inspired by the contrastive unpaired translation framework, we emphasized local features during the enhancing process via contrastive patchwise samples. Moreover, to utilize high-level features in the fundus domain (such as vessels, optic disc/cup, and even lesions), we designed a fundus prior loss to avoid information modification and over-enhancement. Besides, we presented an efficient network architecture to overcome the high consumption in terms of both time and GPU-memory. Through both qualitative and quantitative experiments on a public dataset EyeQ, we demonstrated the superior performance of our proposed method.

Precise quantification of the corneal nerve plexus morphology is of great importance in diagnosing peripheral diabetic neuropathy and assessing the progression of various eye-related systemic diseases, wherein segmentation of corneal nerves is an essential component. In this paper, we proposed and validated a novel pipeline for corneal nerve segmentation, comprising corneal confocal microscopy (CCM) image synthesis, image quality enhancement and nerve segmentation. Our goal was to address three major problems existing in most CCM datasets, namely inaccurate annotations, non-uniform illumination and contrast variations. In our synthesis and enhancement steps, we employed multilayer and patchwise contrastive learning based Generative Adversarial Network (GAN) frameworks, which took full advantage of multi-scale local features. Through both qualitative and quantitative experiments on two publicly available CCM datasets, our pipeline has achieved overwhelming enhancement performance compared to several state-of-the-art methods. Moreover, the segmentation results showed that models trained on our synthetic images performed much better than those trained on a real CCM dataset, which clearly identified the effectiveness of our synthesis method. Overall, our proposed pipeline can achieve satisfactory segmentation performance for poor-quality CCM images without using any manual labels and can effectively enhance those images.

Retinal vascular morphometry is an important biomarker of eye-related cardiovascular diseases such as diabetes and hypertension. And retinal vessel segmentation is a fundamental step in fundus image analyses and diagnoses. In recent years, deep learning based networks have achieved superior performance in medical image segmentation. However, for fine vessels or terminal branches, most existing methods tend to miss or under-segment those structures, inducing isolated breakpoints. In this paper, we proposed Bi-Directional ConvLSTM U-Net with Generative Adversarial Training (BLU-GAN), a novel deep learning model based on U-Net that generates precise predictions of retinal vessels combined with generative adversarial training. Bi-directional ConvLSTM, which can better integrate features from different scales through a coarse-to-fine memory mechanism, is employed to non-linearly combine feature maps extracted from encoding path layers and the previous decoding up-convolutional layers and to replace the simple skip-connection used in the original U-Net. Moreover, we use densely connected convolutions in certain layers to strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. Through extensive experiments, BLU-GAN has shown leading performance among the state-of-the-art methods on the DRIVE, STARE, CHASE_DB1 datasets for retinal vessel segmentation.

Manually annotating medical images is extremely expensive, especially for large-scale datasets. Self-supervised contrastive learning has been explored to learn feature representations from unlabeled images. However, unlike natural images, the application of contrastive learning to medical images is relatively limited. In this work, we propose a self-supervised framework, namely lesion-based contrastive learning for automated diabetic retinopathy (DR) grading. Instead of taking entire images as the input in the common contrastive learning scheme, lesion patches are employed to encourage the feature extractor to learn representations that are highly discriminative for DR grading. We also investigate different data augmentation operations in defining our contrastive prediction task. Extensive experiments are conducted on the publicly-accessible dataset EyePACS, demonstrating that our proposed framework performs outstandingly on DR grading in terms of both linear evaluation and transfer capacity evaluation.

Optical coherence tomography angiography (OCTA) is a novel non-invasive imaging technique that allows visualizations of vasculature and foveal avascular zone (FAZ) across retinal layers. Clinical researches suggest that the morphology and contour irregularity of FAZ are important biomarkers of various ocular pathologies. Therefore, precise segmentation of FAZ has great clinical interest. Also, there is no existing research reporting that FAZ features can improve the performance of deep diagnostic classification networks. In this paper, we propose a novel multi-level boundary shape and distance aware joint learning framework, named BSDA-Net, for FAZ segmentation and diagnostic classification from OCTA images. Two auxiliary branches, namely boundary heatmap regression and signed distance map reconstruction branches, are constructed in addition to the segmentation branch to improve the segmentation performance, resulting in more accurate FAZ contours and fewer outliers. Moreover, both low-level and high-level features from the aforementioned three branches, including shape, size, boundary, and signed directional distance map of FAZ, are fused hierarchically with features from the diagnostic classifier. Through extensive experiments, the proposed BSDA-Net is found to yield state-of-the-art segmentation and classification results on the OCTA-500, OCTAGON, and FAZID datasets.

Optical coherence tomography angiography (OCTA) is a recent advance in ophthalmic imaging, which provides detailed visualization of two important anatomical landmarks, namely foveal avascular zone (FAZ) and retinal vessels (RV). Studies have shown that both FAZ and RV play significant roles in the diagnoses of various eye-related diseases. Therefore, accurate segmentation of FAZ and RV from OCTA images is highly in need. However, due to complicated microstructures and inhomogeneous image quality, there is still room for improvement in existing methods. In this paper, we propose a novel and efficient deep learning framework containing two subnetworks for simultaneously segmenting FAZ and RV from en-face OCTA images, named FARGO. For FAZ, we use RV segmentation as an auxiliary task, which may provide supplementary information especially for low-contrast and low-quality OCTA images. A ResNeSt based encoder with split attention and ImageNet pretraining is employed for FAZ segmentation. For RV, we introduce a coarse-to-fine cascaded network composed of a main segmentation model and several small ones for progressive refining. Spatial attention and channel attention modules are utilized for adaptively integrating local features with global dependencies. Through extensive experiments, FARGO is found to yield outstanding segmentation results for both FAZ and RV on the OCTA-500 dataset, performing even better than methods that utilize 3D OCTA volume as an extra input.