Search

Background and purpose: Thrombus red blood cell (RBC) content has been shown to be a significant factor influencing the efficacy of acute ischemic stroke treatment. In this study, our objective was to evaluate the ability of convolutional neural networks (CNNs) to predict ischemic stroke thrombus RBC content using multiparametric MR images. Materials and methods: Retrieved stroke thrombi were scanned ex vivo using a three-dimensional multi-echo gradient echo sequence and histologically analyzed. 188 thrombus R2*, quantitative susceptibility mapping and late-echo GRE magnitude image slices were used to train and test a 3-layer CNN through cross-validation. Data augmentation techniques involving input equalization and random image transformation were employed to improve network performance. The network was assessed for its ability to quantitatively predict RBC content and to classify thrombi into RBC-rich and RBC-poor groups. Results: The CNN predicted thrombus RBC content with an accuracy of 62% (95% CI 48–76%) when trained on the original dataset and improved to 72% (95% CI 60–84%) on the augmented dataset. The network classified thrombi as RBC-rich or poor with an accuracy of 71% (95% CI 58–84%) and an area under the curve of 0.72 (95% CI 0.57–0.87) when trained on the original dataset and improved to 80% (95% CI 69–91%) and 0.84 (95% CI 0.73–0.95), respectively, on the augmented dataset. Conclusions: The CNN was able to accurately predict thrombus RBC content using multiparametric MR images, and could provide a means to guide treatment strategy in acute ischemic stroke. [ABSTRACT FROM AUTHOR]

Based on deep learning, various pan-sharpening models have achieved excellent results. However, most of them adopt simple addition or concatenation operations to merge the information of low spatial resolution multi-spectral (LRMS) images and panchromatic (PAN) images, which may cause a loss of detailed information. To tackle this issue, inspired by capsule networks, we propose a plug-and-play layer named modified dynamic routing layer (MDRL), which modifies the information transmission mode of capsules to effectively fuse LRMS images and PAN images. Concretely, the lower-level capsules are generated by applying transform operation to the features of LRMS images and PAN images, which preserve the spatial location information. Then, the dynamic routing algorithm is modified to adaptively select the lower-level capsules to generate the higher-level capsule features to represent the fusion of LRMS images and PAN images, which can effectively avoid the loss of detailed information. In addition, the previous addition and concatenation operations are illustrated as special cases of our MDRL. Based on MIPSM with addition operations and DRPNN with concatenation operations, two modified dynamic routing models named MDR–MIPSM and MDR–DRPNN are further proposed for pan-sharpening. Extensive experimental results demonstrate that the proposed method can achieve remarkable spectral and spatial quality. [ABSTRACT FROM AUTHOR]

Pansharpening is a special image fusion task of reconstructing a high-resolution multispectral (HRMS) image by integrating a panchromatic (PAN) image of high spatial resolution and a low-resolution multispectral (LRMS) image. To handle such an ill-posed multimodal fusion task, in this article, we propose a novel pansharpening method, referred to as model-driven and data-driven network (MD3Net), which combines model-driven and data-driven approaches. The architecture design of MD3Net is inspired from the traditional model constructed based on domain knowledge and thus making its network topology explainable and its input-output predictable. To further explore the powerful learning ability of deep-learning-based approaches, we introduce the deep prior into the MD3Net as its implicit regularization, thus improving its data adaptability and representation capability. Comprehensive experiments conducted on both reduced and full resolution of several acknowledged datasets have qualitatively and quantitatively verified the superiority of our network compared with a benchmark consisting of several state-of-the-art approaches. The code can be downloaded from https://github.com/YinsongYan/M3DNet. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion (IVIF) expects to obtain images that retain thermal radiation information from infrared images and texture details from visible images. In this paper, a model-based convolutional neural network (CNN) model, referred to as Algorithm Unrolling Image Fusion (AUIF), is proposed to overcome the shortcomings of traditional CNN-based IVIF models. The proposed AUIF model starts with the iterative formulas of two traditional optimization models, which are established to accomplish two-scale decomposition, i.e., separating low-frequency base information and high-frequency detail information from source images. Then the algorithm unrolling is implemented where each iteration is mapped to a CNN layer and each optimization model is transformed into a trainable neural network. Compared with the general network architectures, the proposed framework combines the model-based prior information and is designed more reasonably. After the unrolling operation, our model contains two decomposers (encoders) and an additional reconstructor (decoder). In the training phase, this network is trained to reconstruct the input image. While in the test phase, the base (or detail) decomposed feature maps of infrared/visible images are merged respectively by an extra fusion layer, and then the decoder outputs the fusion image. Qualitative and quantitative comparisons demonstrate the superiority of our model, which can robustly generate fusion images containing highlight targets and legible details, exceeding the state-of-the-art methods. Furthermore, our network has fewer weights and faster speed. [ABSTRACT FROM AUTHOR]

Vortex beam (VB) possessing spatially helical phaseâ€"front has attracted widespread attention in free-space optical communication, etc. However, the spiral phase of VB is susceptible to atmospheric turbulence, and effective retrieval of the distorted conjugate phase is crucial for its practical applications. Herein, a convolutional neural network (CNN) approach to retrieve the phase distribution of VB is experimentally demonstrated. We adopt a spherical wave to interfere with VB for converting its phase information into intensity changes, and construct a CNN model with excellent image processing capabilities to directly extract phaseâ€"front features from the interferogram. Since the interference intensity is correlated with the phaseâ€"front, the CNN model can effectively reconstruct the wavefront of conjugate VB carrying different initial phases from a single interferogram. The results show that the CNN-based phase retrieval method has a loss of 0.1418 in the simulation and a loss of 0.2344 for the experimental data, and remains robust even in turbulence environments. This approach can improve the information acquisition capability for recovering the distorted wavefront and reducing the reliance on traditional inverse retrieval algorithms, which may provide a promising tool to retrieve the spatial phase distributions of VBs. [ABSTRACT FROM AUTHOR]

The deep learning method, especially convolution neural networks (CNNs), has recently made ground-breaking advances on object detection in very-high-resolution (VHR) optical remote sensing images. However, as CNN is originally designed for the classification of natural images, these methods are not very suitable for object detection of remote sensing images. First, current CNN-based approaches have difficulty to deal with objects that have large rotation variation, which is widely existed in optical remote sensing images. Second, the detectors based on CNN only have limited receptive fields and, thus, can hardly utilize the global contextual information that is essential for accurate detection of small targets. To address these two issues, this article proposes a novel deep learning-based object detection framework, including a geometric transform module (GTM) and a global contextual feature fusion module (GCFM). Especially, the GTM combines rotation and flip transformation to deal with the multiangle characteristics of objects. The GCFM uses a spatial attention mechanism to adaptively involve global contextual information in feature maps to improve the recognition and location accuracy of targets. We introduce the two modules into the YOLOv3 framework to achieve end-to-end detection with high performance and efficiency. Comprehensive evaluations on three publicly available object detection data sets demonstrate the excellent performance of the proposed methods. [ABSTRACT FROM AUTHOR]

3D object recognition is one of the most important tasks in 3D data processing, and has been extensively studied recently. Researchers have proposed various 3D recognition methods based on deep learning, among which a class of view-based approaches is a typical one. However, in the view-based methods, the commonly used view pooling layer to fuse multi-view features causes a loss of visual information. To alleviate this problem, in this paper, we construct a novel layer called Dynamic Routing Layer (DRL) by modifying the dynamic routing algorithm of capsule network, to more effectively fuse the features of each view. Concretely, in DRL, we use rearrangement and affine transformation to convert features, then leverage the modified dynamic routing algorithm to adaptively choose the converted features, instead of ignoring all but the most active feature in view pooling layer. We also illustrate that the view pooling layer is a special case of our DRL. In addition, based on DRL, we further present a Dynamic Routing Convolutional Neural Network (DRCNN) for multi-view 3D object recognition. Our experiments on three 3D benchmark datasets show that our proposed DRCNN outperforms many state-of-the-arts, which demonstrates the efficacy of our method. [ABSTRACT FROM AUTHOR]

• A robust neural network approach is experimentally investigated for identification of conjugate orbital angular momentum modes. • The constructed model has higher robustness against turbulence and lower model computational complexity. • The proposed approach can effectively assist in demodulating conventional amplitude- and phase-modulation signals. Orbital angular momentum shift-keying (OAM-SK), which enables modulation of digital signals by rapidly switching OAM modes, is promising in improving the modulation ability and confidentiality. Robust identification of conjugate OAM modes of VBs perturbed by atmospheric turbulence is critical for efficient demodulation of OAM-SK signals. We experimentally investigate a convolutional neural network (CNN) approach to detect OAM modes from diffraction patterns of turbulence-induced vortex beams. By introducing the hybrid-turbulence with variable turbulence strengths, this approach possessed high robustness against turbulence and low computational complexity in conjugate OAM mode identification, achieved 99.53 % mode detection accuracy in the −8 to 8 range for the hybrid-turbulence of C n 2 = 10 - 14 ∼ 10 - 12 m - 2 / 3 . Mapping and encoding a grayscale image to the OAM modes, an OAM-SK communication link was built and the demodulated bit-error-rate (BER) was only 6.85 × 10−3. We show that this approach also can effectively assist in demodulating the conventional modulation signals, promoting the communication transmission capacity. By combining OAM-SK with on–off keying (OOK) and quadrature phase shift-keying (QPSK) to construct joint-modulation links, and OOK (QPSK) signals were successfully demodulated with the BER of 8.2 × 10−9–9.1 × 10−3 (3.34 × 10−6–2.8 × 10−3) with the assistance of CNN. Our results indicate that the proposed robust CNN is not only effective for demodulating OAM-SK signals, but is also compatible with conventional modulation signals, which is very promising for OAM-SK communication applications that require OAM mode identification. [ABSTRACT FROM AUTHOR]