Search

In this work, we present an unsupervised domain adaptation (UDA) method, named Panoptic Domain Adaptive Mask R-CNN (PDAM), for unsupervised instance segmentation in microscopy images. Since there currently lack methods particularly for UDA instance segmentation, we first design a Domain Adaptive Mask R-CNN (DAM) as the baseline, with cross-domain feature alignment at the image and instance levels. In addition to the image- and instance-level domain discrepancy, there also exists domain bias at the semantic level in the contextual information. Next, we, therefore, design a semantic segmentation branch with a domain discriminator to bridge the domain gap at the contextual level. By integrating the semantic- and instance-level feature adaptation, our method aligns the cross-domain features at the panoptic level. Third, we propose a task re-weighting mechanism to assign trade-off weights for the detection and segmentation loss functions. The task re-weighting mechanism solves the domain bias issue by alleviating the task learning for some iterations when the features contain source-specific factors. Furthermore, we design a feature similarity maximization mechanism to facilitate instance-level feature adaptation from the perspective of representational learning. Different from the typical feature alignment methods, our feature similarity maximization mechanism separates the domain-invariant and domain-specific features by enlarging their feature distribution dependency. Experimental results on three UDA instance segmentation scenarios with five datasets demonstrate the effectiveness of our proposed PDAM method, which outperforms state-of-the-art UDA methods by a large margin.

The automatic reconstruction of single neuron cells from microscopic images is essential to establishing the research on neuron morphology. However, the performance of single neuron reconstruction algorithms is constrained by both the quantity and the quality of the annotated 3D microscopic images since annotating large-scale single neuron models is highly labour intensive. We propose a framework for synthesizing microscopy-realistic 3D neuron images from simulated single neuron skeletons using conditional Generative Adversarial Networks (cGAN). We build the generator network with multi-resolution sub-modules to improve the output fidelity. We evaluate our framework on Janelia-Fly dataset from the BigNeuron project. With both qualitative and quantitative analysis, we show that the proposed framework outperforms the other state-of-the-art methods regarding the quality of the synthetic neuron images. We also show that combining the real neuron images and the synthetic images generated from our framework can improve the performance of neuron segmentation.

U-Net has become one of the state-of-the-art deep learning-based approaches for modern computer vision tasks such as semantic segmentation, super resolution, image denoising, and inpainting. Previous extensions of U-Net have focused mainly on the modification of its existing building blocks or the development of new functional modules for performance gains. As a result, these variants usually lead to an unneglectable increase in model complexity. To tackle this issue in such U-Net variants, in this paper, we present a novel Bi-directional O-shape network (BiO-Net) that reuses the building blocks in a recurrent manner without introducing any extra parameters. Our proposed bi-directional skip connections can be directly adopted into any encoder-decoder architecture to further enhance its capabilities in various task domains. We evaluated our method on various medical image analysis tasks and the results show that our BiO-Net significantly outperforms the vanilla U-Net as well as other state-of-the-art methods. Our code is available at https://github.com/tiangexiang/BiO-Net.

Unsupervised domain adaptation (UDA) for nuclei instance segmentation is important for digital pathology, as it alleviates the burden of labor-intensive annotation and domain shift across datasets. In this work, we propose a Cycle Consistency Panoptic Domain Adaptive Mask R-CNN (CyC-PDAM) architecture for unsupervised nuclei segmentation in histopathology images, by learning from fluorescence microscopy images. More specifically, we first propose a nuclei inpainting mechanism to remove the auxiliary generated objects in the synthesized images. Secondly, a semantic branch with a domain discriminator is designed to achieve panoptic-level domain adaptation. Thirdly, in order to avoid the influence of the source-biased features, we propose a task re-weighting mechanism to dynamically add trade-off weights for the task-specific loss functions. Experimental results on three datasets indicate that our proposed method outperforms state-of-the-art UDA methods significantly, and demonstrates a similar performance as fully supervised methods., Comment: Accepted by CVPR2020

Automated detection and segmentation of individual nuclei in histopathology images is important for cancer diagnosis and prognosis. Due to the high variability of nuclei appearances and numerous overlapping objects, this task still remains challenging. Deep learning based semantic and instance segmentation models have been proposed to address the challenges, but these methods tend to concentrate on either the global or local features and hence still suffer from information loss. In this work, we propose a panoptic segmentation model which incorporates an auxiliary semantic segmentation branch with the instance branch to integrate global and local features. Furthermore, we design a feature map fusion mechanism in the instance branch and a new mask generator to prevent information loss. Experimental results on three different histopathology datasets demonstrate that our method outperforms the state-of-the-art nuclei segmentation methods and popular semantic and instance segmentation models by a large margin.

The evaluation of the flight performance via multiple physiological signals is an important problem in the field of flight safety. A hybrid prediction model is proposed to dispose multiple physiological signals with high dimension in this paper. Main contribution of our model is that of a novel bacterial foraging algorithm (BF) to optimize Elman neural network, which can perform parallel search and escape local minimum easily, and provide better prediction accuracy of the flight performance. Other bio-inspired algorithms such as genetic algorithm (GA), particle swarm optimization (PSO) and chaotic GA are also used to optimize the unknown parameters of the Elman network. Experimental results indicate that the proposed hybrid model based on BF algorithm and Elman network is well suited for the evaluation and prediction of the flight performance. Compared with the other public algorithms, the BF can easily identify the unknown parameters of the established models and has better optimization capability.

Due to the complex morphology of fine vessels, it remains challenging for most of existing models to accurately segment them, particularly the capillaries in color fundus retinal images. In this paper, we propose a novel and lightweight deep learning model called Vessel-Net for retinal vessel segmentation. First, we design an efficient inception-residual convolutional block to combine the advantages of the Inception model and residual module for improved feature representation. Next, we embed the inception-residual blocks inside a U-like encoder-decoder architecture for vessel segmentation. Then, we introduce four supervision paths, including the traditional supervision path, a richer feature supervision path, and two multi-scale supervision paths to preserve the rich and multi-scale deep features during model optimization. We evaluated our Vessel-Net against several recent methods on two benchmark retinal databases and achieved the new state-of-the-art performance (i.e. AUC of 98.21%/98.60% on the DRIVE and CHASE databases, respectively). Our ablation studies also demonstrate that the proposed inception-residual block and the multi-path supervision both can produce impressive performance gains for retinal vessel segmentation.

Structural analysis of neurons can provide valuable insights of brain function. Semantic segmentation of neurons thus becomes an important technique in bioinformatics. Deep learning approaches have shown promising performance in various semantic segmentation problems. However, segmentation of neurons in Electron Microscopy (EM) images has some differences compared with typical segmentation tasks due to the image noise and the disturbance of the intracellular structures. In our work, we propose a network with a ResNet encoder and densely connected decoder with large kernels, and then refinement with simple morphological post-possessing. Two main advantages of our method are: 1) the network can prevent the loss of high-resolution information and enlarge the reception field; 2) the post-processing method is simple and can be directly applied to the probability map from the network to enhance the unconfident area. Evaluated on the ISBI2012 EM membrane segmentation challenge, the proposed method achieves competitive performance.

2D neuron membrane segmentation for Electron Microscopy (EM) images is a key step in the 3D neuron reconstruction task. Compared with the semantic segmentation tasks for general images, the boundary segmentation in EM images is more challenging. In EM segmentation tasks, we need not only to segment the ambiguous membrane boundaries from bubble-like noise in the images, but also to remove shadow-like intracellular structure. In order to address these problems, we propose a Large Kernel Refine Fusion Net, an encoder-decoder architecture with fusion of features at multiple resolution levels. We incorporate large convolutional blocks to ensure the valid receptive fields for the feature maps are large enough, which can reduce information loss. Our model can also process the background together with the membrane boundary by using residual cascade pooling blocks. In addition, the postprocessing method in our work is simple but effective for a final refinement of the output probability map. Our method was evaluated and achieved competitive performances on two EM membrane segmentation tasks: ISBI2012 EM segmentation challenge and mouse piriform cortex segmentation task.

Brain tumor segmentation in magnetic resonance images is a key step for brain cancer diagnosis and clinical treatment. Recently, deep convolutional neural network (DNN) based models have become a popular and effective choice due to their learning capability with a large amount of parameters. However, in traditional 3D DNN models, the valid receptive fields are not large enough for global details from the objective and the large amount of parameters are easy to cause high computational cost and model overfitting. In order to address these problems, we propose a 3D large kernel anisotropic network. In our model, the large kernels in the decoders ensure the valid receptive field is large enough and the anisotropic convolutional blocks in the encoders simulate the traditional isotropic ones with fewer parameters. Our proposed model is evaluated on datasets from the MICCAI BRATS 17 challenge and outperforms several popular 3D DNN architectures.