Search

In the domain of Computer-Aided Diagnosis (CAD) systems, the accurate identification of cancer lesions is paramount, given the life-threatening nature of cancer and the complexities inherent in its manifestation. This task is particularly arduous due to the often vague boundaries of cancerous regions, compounded by the presence of noise and the heterogeneity in the appearance of lesions, making precise segmentation a critical yet challenging endeavor. This study introduces an innovative, an iterative feedback mechanism tailored for the nuanced detection of cancer lesions in a variety of medical imaging modalities, offering a refining phase to adjust detection results. The core of our approach is the elimination of the need for an initial segmentation mask, a common limitation in iterative-based segmentation methods. Instead, we utilize a novel system where the feedback for refining segmentation is derived directly from the encoder–decoder architecture of our neural network model. This shift allows for more dynamic and accurate lesion identification. To further enhance the accuracy of our CAD system, we employ a multi-scale feedback attention mechanism to guide and refine predicted mask subsequent iterations. In parallel, we introduce a sophisticated weighted feedback loss function. This function synergistically combines global and iteration-specific loss considerations, thereby refining parameter estimation and improving the overall precision of the segmentation. We conducted comprehensive experiments across three distinct categories of medical imaging: colonoscopy, ultrasonography, and dermoscopic images. The experimental results demonstrate that our method not only competes favorably with but also surpasses current state-of-the-art methods in various scenarios, including both standard and challenging out-of-domain tasks. This evidences the robustness and versatility of our approach in accurately identifying cancer lesions across a spectrum of medical imaging contexts. Our source code can be found at https://github.com/dewamsa/EfficientFeedbackNetwork. • Efficient feedback mechanisms for enhancing cancerous lesion segmentation. • Multi-stage feedback attention modules for calibrating encoder's feature map. • Combinatorial function to control prediction influence on each feedback iteration. • Extensive evaluation on diverse cancerous lesion datasets. [ABSTRACT FROM AUTHOR]

The deep neural network model based on spatial pyramid pooling for multi-scale context information has been widely adopted in image semantic segmentation. However, this structure reduces the feature resolution during the inference process, which should be recovered by bilinear interpolation. We find that the performance of boundary location heavily depends on the interpolation operation. Assuming that the foreground object has a boundary with uneven curvature changes, it will be difficult to preserve details in the lower resolution feature map. In this case, if the model achieves near-optimal fitting, the prediction of the boundary point needs to satisfy strict constraints. But this condition is very difficult to meet for the model. For this purpose, we propose a constraint balance factor (CBF) relaxation method to improve the fitting ability of the model. The constraint balance factor is learned end-to-end by a sub-network. This sub-network can be easily embedded into a semantic segmentation network that requires interpolation without too many extra parameter increments. The experimental results illustrate that the proposed CBFNet performs well on PASCAL VOC 2012 and Cityscapes datasets. [ABSTRACT FROM AUTHOR]

Extracting brain parenchyma from computed tomography (CT) images of the head is an important prerequisite step in a number of image processing applications, as it improves the computational speed and accuracy of quantitative analyses and image co-registration. In this study, we present a robust method based on fully convolutional neural networks (CNN) to remove non-brain tissues from head CT scans in a computationally efficient manner. The method includes an encoding part, which has sequential convolutional filters that produce feature representation of the input image in low dimensional space, and a decoding part, which consists of convolutional filters that reconstruct the input image from the reduced representation. We trained several CNN models on 122 volumetric head CT scans and tested our models on 22 withheld volumetric CT head scans based on two experts' manual brain segmentation. The performance of our best CNN model on the test set is: Dice Coefficient = 0.998 ± 0.001 (mean ± standard deviation), recall = 0.999 ± 0.001, precision = 0.998 ± 0.001, and accuracy = 1. Our method extracts complete volumetric brain from head CT images in about 2 s which is substantially faster than currently available methods. To the best of our knowledge, this is the first study using CNN to perform brain extraction from CT images. In conclusion, the proposed approach based on CNN provides accurate extraction of brain tissue from head CT images in a computationally efficient manner. [ABSTRACT FROM AUTHOR]

Left ventricle (LV) segmentation plays an important role in the diagnosis of cardiovascular diseases. The cardiac contractile function can be quantified by measuring the segmentation results of LVs. Fully convolutional networks (FCNs) have been proven to be able to segment images. However, a large number of annotated images are required to train the network to avoid overfitting, which is a challenge for LV segmentation owing to the limited small number of available training samples. In this paper, we analyze the influence of augmenting training samples used in an FCN for LV segmentation, and propose a data augmentation approach based on shape models to train the FCN from a few samples. We show that the balanced training samples affect the performance of FCNs greatly. Experiments on four public datasets demonstrate that the FCN trained by our augmented data outperforms most existing automated segmentation methods with respect to several commonly used evaluation measures. [ABSTRACT FROM AUTHOR]

• This paper proposes a CNN-based segmentation framework multiscale dense CNN (MDCNN) to automatically segment cerebral vessel in DSA images. • In the training process, a patch selection strategy is put forward to get enough training data from the limited quantity of annotated cerebrovascular DSA images. • In order to verify the validity of the proposed method, we label a DSA cerebrovascular dataset DCVessel. Accurate cerebrovascular segmentation in Digital Subtraction Angiography (DSA) image, as an indispensable step, can help doctors appropriately estimate the degree of cerebrovascular lesions to avoid misdiagnosis. Because of the complexity of cerebrovascular structure and the uneven distribution of contrast media in DSA, automatic segmentation is a challenging task in clinical diagnosis. In recent years, deep convolutional neural networks (CNN) have outperformed the state-of-art methods and shown great potential for medical image segmentation. This paper proposes a CNN-based segmentation framework Multiscale Dense CNN (MDCNN) to automatically segment cerebral vessel in DSA images. Inspired by U-net, this proposed MDCNN structure is designed as encoder-decoder architecture. Considering that the diameters of blood vessel in cerebrovascular are various, we define a multiscale module to segment cerebral vessel with different diameters. Meanwhile, we redesign the skip connections between encoder and decoder stage to utilize more features from encoder stage. To improve the capability of extracting high-level features, improved dense blocks are introduced. In terms of simplifying parameters, we refer to the idea of deep supervision to make pruning possible. The proposed framework is tested on DCVessel (a DSA cerebrovascular dataset made by our lab). Our proposed method reaches 0.8813, 0.9784, 0.8775, 0.9886, 0.9944 in F1 score, accuracy (Acc), sensitivity (Sen), specificity (Spe) and AUC respectively, outperforming the state-of-art methods. Meanwhile, we evaluate our MDCNN framework on benchmark retinal vessel dataset DRIVE. The promising experiment results demonstrate that proposed MDCNN model is extensible for various vessel segmentation tasks. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes an object detection model named Feature Enriching Object Detection Framework with Weak Segmentation Loss (FEOD) based on convolutional neural networks, which is an improvement of You Only Look Once (YOLO). To overcome the shortcoming of insensitivity to small objects, a novel feature enriching module is proposed to augment the semantic information of the feature in the shallow layer within a typical deep detector. Meanwhile, Focal Loss is also introduced to our model to further improve the algorithm performance. To obtain features more suitable for object detection, a more powerful feature extractor – DetNet is used as the backbone. An mAP of 83.8 in VOC2007 test and an mAP of 82.1 in VOC2012 test are achieved with a FPS of 11.3 with a NVIDIA GTX1080Ti GPU. For a lower resolution version, we achieve an mAP of 81.9 and 80.9 respectively in VOC2007 and VOC2012 tests with FPS of 17. Comparisons with other object detection algorithms have shown that our method works well and achieves a trade-off of accuracy and speed. [ABSTRACT FROM AUTHOR]

Abstract ColorCheckers are reference standards that professional photographers and filmmakers use to ensure predictable results under every lighting condition. The objective of this work is to propose a new fast and robust method for automatic ColorChecker detection. The process is divided into two steps: (1) ColorCheckers localization and (2) ColorChecker patches recognition. For the ColorChecker localization, we trained a detection convolutional neural network using synthetic images. The synthetic images are created with the 3D models of the ColorChecker and different background images. The output of the neural networks are the bounding box of each possible ColorChecker candidates in the input image. Each bounding box defines a cropped image which is evaluated by a recognition system, and each image is canonized with regards to color and dimensions. Subsequently, all possible color patches are extracted and grouped with respect to the center's distance. Each group is evaluated as a candidate for a ColorChecker part, and its position in the scene is estimated. Finally, a cost function is applied to evaluate the accuracy of the estimation. The method is tested using real and synthetic images. The proposed method is fast, robust to overlaps and invariant to affine projections. The algorithm also performs well in case of multiple ColorCheckers detection. [ABSTRACT FROM AUTHOR]

Abstract Deep convolutional neural networks (DCNNs) have been used to achieve state-of-the-art performance on many computer vision tasks (e.g., object recognition, object detection, semantic segmentation) thanks to a large repository of annotated image data. Large labeled datasets for other sensor modalities, e.g., multispectral imagery (MSI), are not available due to the large cost and manpower required. In this paper, we adapt state-of-the-art DCNN frameworks in computer vision for semantic segmentation for MSI imagery. To overcome label scarcity for MSI data, we substitute real MSI for generated synthetic MSI in order to initialize a DCNN framework. We evaluate our network initialization scheme on the new RIT-18 dataset that we present in this paper. This dataset contains very-high resolution MSI collected by an unmanned aircraft system. The models initialized with synthetic imagery were less prone to over-fitting and provide a state-of-the-art baseline for future work. [ABSTRACT FROM AUTHOR]

Retinal vessel analysis of fundus images is an indispensable method for the screening and diagnosis of related diseases. In this paper, we propose a novel retinal vessel segmentation method of the fundus images based on convolutional neural network (CNN) and fully connected conditional random fields (CRFs). The segmentation process is mainly divided into two steps. Firstly, a multiscale CNN architecture with an improved cross-entropy loss function is proposed to produce the probability map from image to image. We construct the multiscale network by combining the feature map of each middle layer to learn more detail information of the retinal vessels. Meanwhile, our proposed cross-entropy loss function ignores the slightest loss of relatively easy samples in order to take more attention to learn the hard examples. Secondly, CRFs is applied to get the final binary segmentation result which makes use of more spatial context information by taking into account the interactions among all of the pixels in the fundus images. The effectiveness of the proposed method has been evaluated on two public datasets, i.g., DRIVE and STARE with comparisons against eleven state-of-the-art approaches including five deep learning based methods. Results show that our method allows for detection of more tiny blood vessels and more precise locating of the edges. [ABSTRACT FROM AUTHOR]

Lung cancer has been recognized as the primary global cause of death among cancer patients. This work is intended to develop a methodology for diagnosis of lung nodules using images from the Lung Image Database Consortium and Image Database Resource Initiative. The proposed method uses image processing and pattern recognition techniques. To differentiate the patterns of malignant and benign forms, we used index basic taxic weights and standardized taxic weights. Finally, we applied a convolutional neural network for classification. In the test stage, we applied the proposed methodology to 50,580 (14,184 malignant and 36,396 benign) nodules from the image database. The proposed method presents promising results for the diagnosis of malignancy and benignity, achieving an accuracy of 92.63%, sensitivity of 90.7%, specificity of 93.47%, and receiver operating characteristic curve of 0.934. These results are promising and demonstrate a real rate of correct detections using the texture features. Because precocious detection allows faster therapeutic intervention, and thus a more favorable prognosis for the patient, we propose herein a methodology that contributes to the field in this aspect. [ABSTRACT FROM AUTHOR]

Highlights • We propose a novel two-stream ConvNet model for first person action recognition. • Our model effectively captures the temporal structure of actions. • We introduce novel long-term pooling operators for appearance and motion information. • We analyze the effect of the proposed architecture with extensive experiments. • We achieve the state-of-the-art performance for first person action recognition. Abstract First person action recognition is an active research area with increasingly popular wearable devices. Action classification for first person video (FPV) is more challenging than conventional action classification due to strong egocentric motions, frequent changes of viewpoints, and diverse global motion patterns. To tackle these challenges, we introduce a two-stream convolutional neural network that improves action recognition via long-term fusion pooling operators. The proposed method effectively captures the temporal structure of actions by leveraging a series of frame-wise features of both appearance and motion in actions. Our experiments validate the effect of the feature pooling operators, and show that the proposed method achieves state-of-the-art performance on standard action datasets. [ABSTRACT FROM AUTHOR]

Biometric technology based on human gait identifies humans at a far distance even if the individual's face is covered, hidden, or not visible to cameras in dark environments. Previous studies based on human gait were conducted considering both bright and dark environments for human identification in surveillance systems. The studies conducted in low-illumination environments (dark environments) are based on side view images (horizontal walking) of subjects. However, there are cases in which people only show the front and back views of their bodies while they are walking in low-illumination corridors. In these views, it is difficult to identify humans by using conventional features such as cycle, cadence, stride length of walking, and distance between points (ankle, knee, and hip). Additionally, the cases of problems such as people carrying cellphones and/or small personal items (a purse, bag, clothes, etc.) have critical effects on the accuracy of human identification. To overcome these problems, we propose a new human identification technique, which is based on the front and back view images of a human, captured by using a thermal camera sensor. Our technique uses movements of the human body for identification, particularly movement of the head, shoulders, and legs. We have used a convolutional neural network for feature extraction and classification in this study. Five datasets were compiled by collecting data of 80 people including men and women in both bright and dark environments. The experimental results with our collected data and open database showed a higher performance by using our method compared to those of previous studies. [ABSTRACT FROM AUTHOR]

We propose an adaptation of a convolutional neural network (CNN) scheme proposed for segmenting brain lesions with considerable mass-effect, to segment white matter hyperintensities (WMH) characteristic of brains with none or mild vascular pathology in routine clinical brain magnetic resonance images (MRI). This is a rather difficult segmentation problem because of the small area ( i . e ., volume) of the WMH and their similarity to non-pathological brain tissue. We investigate the effectiveness of the 2D CNN scheme by comparing its performance against those obtained from another deep learning approach: Deep Boltzmann Machine (DBM), two conventional machine learning approaches: Support Vector Machine (SVM) and Random Forest (RF), and a public toolbox: Lesion Segmentation Tool (LST), all reported to be useful for segmenting WMH in MRI. We also introduce a way to incorporate spatial information in convolution level of CNN for WMH segmentation named global spatial information (GSI). Analysis of covariance corroborated known associations between WMH progression, as assessed by all methods evaluated, and demographic and clinical data. Deep learning algorithms outperform conventional machine learning algorithms by excluding MRI artefacts and pathologies that appear similar to WMH. Our proposed approach of incorporating GSI also successfully helped CNN to achieve better automatic WMH segmentation regardless of network's settings tested. The mean Dice Similarity Coefficient (DSC) values for LST-LGA, SVM, RF, DBM, CNN and CNN-GSI were 0.2963, 0.1194, 0.1633, 0.3264, 0.5359 and 5389 respectively. [ABSTRACT FROM AUTHOR]

Diagnostic information regarding the health status of the corneal endothelium may be obtained by analyzing the size and the shape of the endothelial cells in specular microscopy images. Prior to the analysis, the endothelial cells need to be extracted from the image. Up to today, this has been performed manually or semi-automatically. Several approaches to automatic segmentation of endothelial cells exist; however, none of them is perfect. Therefore this paper proposes to perform cell segmentation using a U-Net-based convolutional neural network. Particularly, the network is trained to discriminate pixels located at the borders between cells. The edge probability map outputted by the network is next binarized and skeletonized in order to obtain one-pixel wide edges. The proposed solution was tested on a dataset consisting of 30 corneal endothelial images presenting cells of different sizes, achieving an AUROC level of 0.92. The resulting DICE is on average equal to 0.86, which is a good result, regarding the thickness of the compared edges. The corresponding mean absolute percentage error of cell number is at the level of 4.5% which confirms the high accuracy of the proposed approach. The resulting cell edges are well aligned to the ground truths and require a limited number of manual corrections. This also results in accurate values of the cell morphometric parameters. The corresponding errors range from 5.2% for endothelial cell density, through 6.2% for cell hexagonality to 11.93% for the coefficient of variation of the cell size. [ABSTRACT FROM AUTHOR]

A novel method for the classification and retrieval of 3D models is proposed; it exploits the 2D panoramic view representation of 3D models as input to an ensemble of convolutional neural networks which automatically compute the features. The first step of the proposed pipeline, pose normalization is performed using the SYMPAN method, which is also computed on the panoramic view representation. In the training phase, three panoramic views corresponding to the major axes, are used for the training of an ensemble of convolutional neural networks. the panoramic views consist of 3-channel images, containing the Spatial Distribution Map, the Normals’ Deviation Map and the magnitude of the Normals’ Devation Map Gradient Image. The proposed method aims at capturing feature continuity of 3D models, while simultaneously minimizing data preprocessing via the construction of an augmented image representation. It is extensively tested in terms of classification and retrieval accuracy on two standard large scale datasets: ModelNet and ShapeNet. [ABSTRACT FROM AUTHOR]

Dynamic Textures (DTs) are sequences of images of moving scenes that exhibit certain stationarity properties in time such as smoke, vegetation and fire. The analysis of DT is important for recognition, segmentation, synthesis or retrieval for a range of applications including surveillance, medical imaging and remote sensing. Convolutional Neural Networks (CNNs) have recently proven to be well suited for texture analysis with a design similar to filter banks. We develop a new DT analysis method based on a CNN method applied on three orthogonal planes. We train CNNs on spatial frames and temporal slices extracted from the DT sequences and combine their outputs to obtain a competitive DT classifier trained end-to-end. Our results on a wide range of commonly used DT classification benchmark datasets prove the robustness of our approach. Significant improvement of the state of the art is shown on the larger datasets. [ABSTRACT FROM AUTHOR]

Multi-label image classification is a challenging problem in computer vision. Motivated by the recent development in image classification performance using Deep Neural Networks, in this work, we propose a flexible deep Convolutional Neural Network (CNN) framework, called Local-Global-CNN (LGC), to improve multi-label image classification performance. LGC consists of firstly a local level multi-label classifier which takes object segment hypotheses as inputs to a local CNN. The output results of these local hypotheses are aggregated together with max-pooling and then re-weighted to consider the label co-occurrence or interdependencies information by using a graphical model in the label space. LGC also utilizes a global CNN that is trained by multi-label images to directly predict the multiple labels from the input. The predictions of local and global level classifiers are finally fused together to obtain MAP estimation of the final multi-label prediction. The above LGC framework could benefit from a pre-train process with a large-scale single-label image dataset, e.g., ImageNet. Experimental results have shown that the proposed framework could achieve promising performance on Pascal VOC2007 and VOC2012 multi-label image dataset. [ABSTRACT FROM AUTHOR]

Synthetic aperture radar (SAR) imaging system is usually an observation of the earths' surface. It means that rich structures exist in SAR images. Convolutional neural network (CNN) is good at learning features from raw data automatically, especially the structural features. Inspired by these, we propose a novel SAR image segmentation method based on convolutional-wavelet neural networks (CWNN) and Markov Random Field (MRF). In this approach, a wavelet constrained pooling layer is designed to replace the conventional pooling in CNN. The new architecture can suppress the noise and is better at keeping the structures of the learned features, which are crucial to the segmentation tasks. CWNN produces the segmentation map by patch-by-patch scanning. The segmentation result of CWNN will be used with two labeling strategies (i.e., a superpixel approach and a MRF approach) to produce the final segmentation map. The superpixel approach is used to enforce the smooth nature on the local region. On the other hand, the MRF approach is used to preserve the edges and the details of the SAR image. Specifically, two segmentation maps will be produced by applying the superpixel and MRF approaches. The first segmentation map is obtained by combining the segmentation map of CWNN and the superpixel approach, and the second segmentation map is obtained by applying the MRF approach on the original SAR image. Afterwards, these two segmentation maps are fused by using the sketch map of the SAR image to produce the final segmentation map. Experiments on the texture images demonstrate that the CWNN is effective for the segmentation tasks. Moreover, the experiments on the real SAR images show that our approach obtains the regions with labeling consistency and preserves the edges and details at the same time. [ABSTRACT FROM AUTHOR]

Capsule endoscopy (CE) is the first-line diagnostic tool for inspecting gastrointestinal (GI) tract diseases. It is a tremendous task on examining and managing the CE videos by endoscopists. Therefore, a computer-aided diagnosis system is desired and urgent. In this paper, a general cascaded spatial–temporal deep framework is proposed to understand the most commonly seen contents of whole GI tract videos. First, the noisy contents such as feces, bile, bubble, and low power images are detected and removed by a Convolutional Neural Network (CNN) model. The clear images are then classified into entrance, stomach, small intestine, and colon by the second CNN. Finally, the topographic segmentation of the whole video is performed with a global temporal integration strategy by Hidden Markov Model (HMM). Compared to existing methods, the proposed framework performs noise content detection and topographic segmentation at the same time, which significantly reduces the number of images to be checked by endoscopists and segments images of different organs more accurately. Experiments on a dataset with 630K images from 14 patients demonstrate that the proposed approach achieves a promising performance in terms of effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Scene recognition plays an important role in the task of visual information retrieval, segmentation and image/video understanding. Traditional approaches for scene recognition usually utilize handcrafted features and have the drawbacks of poor representation ability, which can be improved by employing deep convolutional neural network (CNN) features that contain more semantic and structure information and thus possess more discriminative ability via multiple linear and non-linear transformations. However, an amount of detailed information may be lost when only the final output features which have gone through a certain number of transformations are applied to scene recognition. The features which are generated from the intermediate layers are not fully utilized. In this work, the GoogLeNet model is employed and divided into three parts of layers from bottom to top. The output features from each of the three parts are applied for scene recognition, which leads to the proposed GoogLeNet based multi-stage feature fusion (G-MS2F). What's more, the product rule is used to generate the final decision for scene recognition from the three outputs corresponding to the three parts of the proposed model. The experimental results demonstrate that the proposed model is superior to a number of state-of-the-art CNN models for scene recognition, and obtains the recognition accuracy of 92.90%, 79.63% and 64.06% on the benchmark scene recognition datasets Scene15, MIT67 and SUN397, respectively. [ABSTRACT FROM AUTHOR]

We present new achievements on the use of deep convolutional neural networks (CNN) in the problem of pedestrian detection (PD). In this paper, we aim to address the following questions: (i) Given non-deep state-of-the-art pedestrian detectors (e.g. ACF, LDCF), is it possible to improve their top performances? ; (ii) is it possible to apply a pre-trained deep model to these detectors to boost their performances in the PD context? In this paper, we address the aforementioned questions by cascading CNN models (pre-trained on Imagenet) with state-of-the-art non-deep pedestrian detectors. Furthermore, we also show that this strategy is extensible to different segmentation maps (e.g. RGB, gradient, LUV) computed from the same pedestrian bounding box (i.e. the proposal). We demonstrate that the proposed approach is able to boost the detection performance of state-of-the-art non-deep pedestrian detectors. We apply the proposed methodology to address the pedestrian detection problem on the publicly available datasets INRIA and Caltech. [ABSTRACT FROM AUTHOR]

• A patch-based, residual, asymmetric, encoder-decoder 2D CNN architecture with 11 convolutional layers and 84,217 trainable parameters is introduced. • Utilized a training strategy combining the Focal Tversky and Binary cross-entropy loss functions to overcome the class imbalance issue in CT images. • A voting mechanism is applied to the result predicted by the weight matrices to ensure stable results. • Compared the method's performance on the testing dataset with that of selected high-performing methods from the ISLES 2018 challenge and achieved the second rank. Background and objectives: This paper has introduced a patch-based, residual, asymmetric, encoder-decoder CNN that solves two major problems in acute ischemic stroke lesion segmentation from CT and CT perfusion data using deep neural networks. First, the class imbalance is encountered since the lesion core size covers less than 5% of the volume of the entire brain. Second, deeper neural networks face the drawback of vanishing gradients, and this degrades the learning ability of the network. Methods: The neural network architecture has been designed for better convergence and faster inference time without compromising performance to address these difficulties. It uses a training strategy combining Focal Tversky and Binary cross-entropy loss functions to overcome the class imbalance issue. The model comprises only four resolution steps with a total of 11 convolutional layers. A base filter of 8, used for the residual connection with two convolutional blocks at the encoder side, is doubled after each resolution step. Simultaneously, the decoder consists of residual blocks with one convolutional layer and a constant number of 8 filters in each resolution step. This proposition allows for a lighter build with fewer trainable parameters as well as aids in avoiding overfitting by allowing the decoder to decode only necessary information. Results: The presented method has been evaluated through submission on the publicly accessible platform of the Ischemic Stroke Lesion Segmentation (ISLES) 2018 medical image segmentation challenge achieving the second-highest testing dice similarity coefficient (DSC). The experimental results demonstrate that the proposed model achieves comparable performance to other submitted strategies in terms of DSC Precision, Recall, and Absolute Volume Difference (AVD). Conclusions: Through the proposed approach, the two major research gaps are coherently addressed while achieving high challenge scores by solving the mentioned problems. Our model can serve as a tool for clinicians and radiologists to hasten decision-making and detect strokes efficiently. [ABSTRACT FROM AUTHOR]

Scene text detection and scene segmentation are meaningful tasks in the computer vision field. Could the semantic scene segmentation assist scene text detection in any degree? For example, can we expect the probability of a region being text is low if its surrounding segment, i.e., its context, is labeled as sky? In this paper, we have a positive answer by constructing a scene context-based text detection model. In this model, we use texton features and a fully-connected conditional random field (CRF) to estimate pixel-level scene class׳s probability to be considered as image׳s context feature. Meanwhile, maximally stable extremal regions (MSERs) are extracted, integrated and extended as image patches of character candidates. Then, each image patch is fed to a simple two-layer convolutional neural network (CNN) to automatically extract its character feature. The averaged context feature of the corresponding patch is considered as the patch׳s context feature. The character feature and context feature are fused as the input into a support vector machine for text/non-text determination. Finally, as a post-processing, neighboring text regions are grouped hierarchically. The performance evaluation on ICDAR2013 and SVT databases, as well as a preliminary evaluation on a patch-level database, proves that the scene context can improve the performance of scene text detection. Moreover, the comparative study with state-of-the-art methods shows the top-level performance of our method. [ABSTRACT FROM AUTHOR]

Segmentation of retinal blood vessels is of substantial clinical importance for diagnoses of many diseases, such as diabetic retinopathy, hypertension and cardiovascular diseases. In this paper, the supervised method is presented to tackle the problem of retinal blood vessel segmentation, which combines two superior classifiers: Convolutional Neural Network (CNN) and Random Forest (RF). In this method, CNN performs as a trainable hierarchical feature extractor and ensemble RFs work as a trainable classifier. By integrating the merits of feature learning and traditional classifier, the proposed method is able to automatically learn features from the raw images and predict the patterns. Extensive experiments have been conducted on two public retinal images databases (DRIVE and STARE), and comparisons with other major studies on the same database demonstrate the promising performance and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Recently, deep convolutional neural networks have achieved great success for medical image segmentation. However, unlike segmentation of natural images, most medical images such as MRI and CT are volumetric data. In order to make full use of volumetric information, 3D CNNs are widely used. However, 3D CNNs suffer from higher inference time and computation cost, which hinders their further clinical applications. Additionally, with the increased number of parameters, the risk of overfitting is higher, especially for medical images where data and annotations are expensive to acquire. To issue this problem, many 2.5D segmentation methods have been proposed to make use of volumetric spatial information with less computation cost. Despite these works lead to improvements on a variety of segmentation tasks, to the best of our knowledge, there has not previously been a large-scale empirical comparison of these methods. In this paper, we aim to present a review of the latest developments of 2.5D methods for volumetric medical image segmentation. Additionally, to compare the performance and effectiveness of these methods, we provide an empirical study of these methods on three representative segmentation tasks involving different modalities and targets. Our experimental results highlight that 3D CNNs may not always be the best choice. Despite all these 2.5D methods can bring performance gains to 2D baseline, not all the methods hold the benefits on different datasets. We hope the results and conclusions of our study will prove useful for the community on exploring and developing efficient volumetric medical image segmentation methods. • We summarize the latest developments about 2.5D segmentation methods for volumetric medical image segmentation tasks. • We make a large-scale empirical comparison of 2.5D segmentation methods with 2D and 3D methods on three representative public datasets involving different modalities (CT and MRI) and targets (cardiac, prostate and abdomen) to systematically evaluate the performance of these methods on different segmentation tasks. • We make conclusions base on the experimental results which we hope will prove useful for the community on exploring and developing efficient volumetric medical image segmentation methods. [ABSTRACT FROM AUTHOR]

Acoustic steady-state excitation spatial spectroscopy (ASSESS) is a full-field, ultrasonic non-destructive evaluation (NDE) technique used to locate and characterize defects in plate-like structures. ASSESS generates a steady-state, single-tone ultrasonic excitation in a structure and a scanning laser Doppler Vibrometer (LDV) measures the resulting full-field surface velocity response. Traditional processing techniques for ASSESS data rely on wavenumber domain analysis. This paper presents the alternative use of a convolutional neural network (CNN), trained using simulated ASSESS data, to predict the local plate thickness at every pixel in the wavefield measurement directly. The defect detection accuracy of CNN-based thickness predictions are shown to improve for defects of greater size, and for defects with higher thickness reductions. The CNN demonstrates the ability to predict thickness accurately in regions where Lamb wave dispersion relations are complex or unknown, such as near the boundaries of a test specimen, so long as the CNN is trained on data that accounts for these regions. The CNN also shows generalizability to ASSESS experimental data, despite an entirely simulated training dataset. • Convolutional neural network model trained for material thickness prediction. • Model trained purely on simulated steady-state ultrasonic wavefields. • Robust generalizability of model demonstrated on geometries outside of training. • Robust generalizability of model demonstrated on experimental data sets. • Model performance explored with respect to defect size and thickness reduction. [ABSTRACT FROM AUTHOR]

• Proton exchange membrane fuel cell materials 3D structure are captured by micro-CT. • A symmetrical U-ResNet convolutional neural network model is used for segmentation. • CNN segmentation outperforms the manual segmentation, especially for carbon fibre and binder. • CNN-based segmentation captures realistic fuel cell structure parameters. This study demonstrates the benefit of convolutional neural networks to accurately classify the different materials of proton exchange membrane fuel cells using X-ray micro-computed tomography. Nineteen 2D orthoslices from a 3D tomography dataset were segmented with high quality and used to train a novel U-ResNet convolutional neural network (CNN) to segment the complete volume. The results were compared with a 3D manual segmentation performed under time constraints. The CNN segmented all phases with equal or greater accuracy in comparison to the manual segmentation. In particular, the CNN excelled in separating the carbon fibres and binder phase in the gas diffusion layer, which is usually completely avoided due to difficulty. Further, permeability calculations were performed on the binder void space for both segmentations, with the CNN displaying realistic results. Therefore, CNNs have been shown to be a viable and valuable method in segmenting such fuel cells with increased efficiency and accuracy. [ABSTRACT FROM AUTHOR]

Determination of pore space characteristics is important for carbonate reservoir interpretation and evaluation. Field outcrops can reflect the geology of the subsurface reservoir. The most common method of traditional outcrop research is field investigation, which is likely to cause human errors. Digital image recognition of geological outcrop areas can reduce the problem of inaccurate descriptions of geological structures caused by human factors. Automatic extraction based on convolutional neural networks can greatly reduce the amount of such work. By enhancing the deep-learning model of a convolution neural network, the mask region-convolutional neural network (Mask R-CNN) is proposed. This method adapts to the multiscale features of the cavity by manipulating the scales of the input image. To verify the applicability of the model, the method is applied to automatic cavity identification in the digital outcrop profile of the Dengying Formation (2nd Member) in Xianfeng, Ebian. The parameters are calculated layer by layer, and their distribution characteristics are quantitatively analysed, indicating good application results. To verify the advanced nature of the model, the cavity extraction results of this method are compared with those of traditional image segmentation, machine learning and other depth learning methods. Precision analysis shows that the performance of the method proposed in this paper is superior to the traditional methods. In addition, the cavity feature parameters extracted by this method are compared with the manual extraction. The accuracy for the cavity number, surface porosity, and average cavity area is over 80%, 88%, and 72%, respectively for the proposed method. The results show that the proposed method is reliable and accurate, hence it will help to provide a basis for reservoir prediction in the process of regional geological exploration. • A multi-scale regional convolution neural network method for detecting outcrop cavities is proposed. • The method has higher accuracy than the traditional image detection method. • The quantitative characterization and statistical process of void parameters are highly automated. • Automatic extraction of field outcrops provides a basis for carbonate reservoir evaluation on the profile. [ABSTRACT FROM AUTHOR]

[Display omitted] • Predicting urban pluvial flood depth using deep learning. • Generates flood predictions for areas and rain events where U-FLOOD was not trained. • Systematic assessment of required spatial inputs for optimal predictions. • Sensitivity assessment of neural network hyperparameters. This study investigates how deep-learning can be configured to optimise the prediction of 2D maximum water depth maps in urban pluvial flood events. A neural network model is trained to exploit patterns in hyetographs as well as in topographical data, with the specific aim of enabling fast predictions of flood depths for observed rain events and spatial locations that have not been included in the training dataset. A neural network architecture that is widely used for image segmentation (U-NET) is adapted for this purpose. Key novelties are a systematic investigation of which spatial inputs should be provided to the deep learning model, which hyper-parametrization optimizes predictive performance, and a systematic evaluation of prediction performance for locations and rain events that were not considered in training. We find that a spatial input dataset of only 5 variables that describe local terrain shape and imperviousness is optimal to generate predictions of water depth. Neural network architectures with between 97,000 and 260,000,000 parameters are tested, and a model with 28,000,000 parameters is found optimal. U-FLOOD is demonstrated to yield similar predictive performance as existing screening approaches, even though the assessment is performed for natural rain events and in locations unknown to the network, and flood predictions are generated within seconds. Improvements can likely be obtained by ensuring a balanced representation of temporal and spatial rainfall patterns in the training dataset, further improved spatial input datasets, and by linking U-FLOOD to dynamic sewer system models. [ABSTRACT FROM AUTHOR]

[Display omitted] • A pipeline for corneal endothelium image segmentation is proposed. • It combines a marker-based watershed transform and convolutional neural network. • CNN predicts markers for each cell, next used to segment cell edge probability map. • The method was tested on a heterogeneous dataset of corneal endothelium images. • The method correctly detected 97.72% of corneal endothelial cells. Quantitive information about corneal endothelium cells' morphometry is vital for assessing cornea pathologies. Nevertheless, in clinical, everyday routine dominates qualitative assessment based on visual inspection of the microscopy images. Although several systems exist for automatic segmentation of corneal endothelial cells, they exhibit certain limitations. The main one is sensitivity to low contrast and uneven illumination, resulting in over-segmentation. Subsequently, image segmentation results often require manual editing of missing or false cell edges. Therefore, this paper further investigates the problem of corneal endothelium cell segmentation. A fully automatic pipeline is proposed that incorporates the watershed algorithm for marker-driven segmentation of corneal endothelial cells and an encoder-decoder convolutional neural network trained in a sliding window set up to predict the probability of cell centers (markers) and cell borders. The predicted markers are used for watershed segmentation of edge probability maps outputted by a neural network. The proposed method's performance on a heterogeneous dataset comprising four publicly available corneal endothelium image datasets is analyzed. The performance of three convolutional neural network models (i.e., U-Net, SegNet, and W-Net) incorporated in the proposed pipeline is examined. The results of the proposed pipeline are analyzed and compared to the state-of-the-art competitor. The obtained results are promising. Regardless of the convolutional neural model incorporated into the proposed pipeline, it notably outperforms the competitor. The proposed method scored 97.72% of cell detection accuracy, compared to 87.38% achieved by the competitor. The advantage of the introduced method is also apparent for cell size, DICE coefficient, and Modified Hausdorff distance. [ABSTRACT FROM AUTHOR]

• We proposed and tested a novel SG-UNet, which takes multi-resolution input images and achieve multi-resolution prediction under the multi-supervision, to meet the needs of precise biomedical image segmentation. • Based on this architecture, we proposed the zoom strategy to gradually change the loss weight of multi-supervision in the training process, using sub-modules instead of pre-trained models to guide the training of the main model. • We combined the zoom strategy with loss function to gradually enhance the focus training on a sparse set of hard samples, to improve the segmentation results without finetuning. • To the best of our knowledge, this work is the first time that deep learning method is used to segment rectal cancer lesions on CT images, and validated on the publicly available dataset, which offers a baseline performance for future evaluation of this dataset. Background and objective: The deep neural network model can learn complex non-linear relationships in the data and has superior flexibility and adaptability. A downside of this flexibility is that they are sensitive to initial conditions, both in terms of the initial random weights and in terms of the statistical noise in the training dataset. And the disadvantage caused by adaptability is that deep convolutional networks usually have poor robustness or generalization when the models are trained using the extremely limited amount of labeled data, especially in the biomedical imaging informatics field. Methods: In this paper, we propose to develop and test a stacked generalization U-shape network (SG-UNet) based on the zoom strategy applying to biomedical image segmentation. SG-UNet is essentially a stacked generalization architecture consisting of multiple sub-modules, which takes multi-resolution images as input and uses hybrid features to segment regions of interest and detect diseases under the multi-supervision. The proposed new SG-UNet applies the zoom of multi-supervision to do optimization search in global feature space without pre-training. Besides, the zoom loss function can gradually enhance the focus training on a sparse set of hard samples. Results: We evaluated the proposed algorithm in comparison with several popular U-shape ensemble network architectures across multi-modal biomedical image segmentation tasks to segment malignant rectal cancers, polyps and glands from the three imaging modalities of computed tomography (CT), digital colonoscopy and histopathology images. Applying the proposed algorithm improves 3.116%, 2.676%, 2.356% on Dice coefficients, and 3.044%, 2.420%, 1.928% on F2-score for the three imaging modality datasets, respectively. The comparison results using different amounts of rectal cancer CT data show that the proposed algorithm has a slower tendency of diminishing marginal efficiency. And glands segmentation study results also support the feasibility of yielding comparable performance with other state-of-the-art methods. Conclusions: The proposed algorithm can be trained more efficiently by using the small image datasets without using additional techniques such as fine-tuning, and achieves higher accuracy with less computational complexity than other stacked ensemble networks for biomedical image segmentation. [ABSTRACT FROM AUTHOR]

Enclosed spaces are common in built structures but pose a challenge to many forms of manual or robotic surveying and maintenance tasks. Part of this challenge is to train robot systems to understand their environment without human intervention. This paper presents a method to automatically classify features within a closed void using deep learning. Specifically, the paper considers a robot placed under floorboards for the purpose of autonomously surveying the underfloor void. The robot uses images captured using an RGB camera to identify regions such as floorboards, joists, air vents and pipework. The paper first presents a standard mask regions convolutional neural network approach, which gives modest performance. The method is then enhanced using a two-stage transfer learning approach with an existing dataset for interior scenes. The conclusion from this work is that, even with limited training data, it is possible to automatically detect many common features of such areas. • A method for semantic segmentation in closed spaces • Database of underfloor images are captured and segmented using a mask R-CNN. • Two-stage transfer learning and other optimisations presented to maximise accuracy • Case-study application to the detection of floorboards, joists, vents, pipes, etc for automated insulation spraying • Mapping of segmented data labels to 3D point cloud data [ABSTRACT FROM AUTHOR]

Hyperspectral image analysis has been gaining research attention thanks to the current advances in sensor design which have made acquiring such imagery much more affordable. Although there exist various approaches for segmenting hyperspectral images, deep learning has become the mainstream. However, such large-capacity learners are characterized by significant memory footprints. This is a serious obstacle in employing deep neural networks on board a satellite for Earth observation. In this paper, we introduce resource-frugal quantized convolutional neural networks, and greatly reduce their size without adversely affecting the classification capability. Our experiments performed over two hyperspectral benchmarks showed that the quantization process can be seamlessly applied during the training, and it leads to much smaller and still well-generalizing deep models. [ABSTRACT FROM AUTHOR]

• Deep learning-based automatic segmentation of concrete cracks in tunnels. • A new end-to-end crack segmentation method based on fully convolutional networks. • More efficiency and higher accuracy than the conventional and other deep learning-based crack segmentation methods. • Use of dilated convolution, spatial pyramid pooling, skip connections, and an optimized loss function. Automatic detection and segmentation of concrete cracks in tunnels remains a high-priority task for civil engineers. Image-based crack segmentation is an effective method for crack detection in tunnels. With the development of deep learning techniques, especially the development of image segmentation based on convolutional neural networks, new opportunities have been brought to crack detection. In this study, an improved deep fully convolutional neural network, named as CrackSegNet, is proposed to conduct dense pixel-wise crack segmentation. The proposed network consists of a backbone network, dilated convolution, spatial pyramid pooling, and skip connection modules. These modules can be used for efficient multiscale feature extraction, aggregation, and resolution reconstruction which greatly enhance the overall crack segmentation ability of the network. Compared to the conventional image processing and other deep learning-based crack segmentation methods, the proposed network shows significantly higher accuracy and generalization, making tunnel inspection and monitoring highly efficient, low cost, and eventually automatable. [ABSTRACT FROM AUTHOR]

• Malposition of the acetabular component cause dislocation and prosthetic impingement after total hip arthroplasty, which significantly affect postoperative quality of life and implant longevity. • We introduce a new method for accurate estimation of functional PSI without requiring CT image in order to lower radiation exposure of the patient which opens up the possibility of increasing its application in a larger number of hospitals where CT is not acquired in a routine protocol. • We investigate the Mask R-CNN performance on radiography image segmentation and compare it with U-Net that widely is used in biomedical tasks. • The benefit of using transfer learning, multi-task learning and data augmentation is investigated. Malposition of the acetabular component causes dislocation and prosthetic impingement after Total Hip Arthroplasty (THA), which significantly affects the postoperative quality of life and implant longevity. The position of the acetabular component is determined by the Pelvic Sagittal Inclination (PSI), which not only varies among different people but also changes in different positions. It is important to recognize individual dynamic changes of the PSI for patient-specific planning of the THA. Previously PSI was estimated by registering the CT and radiography images. In this study, we introduce a new method for accurate estimation of functional PSI without requiring CT image in order to lower radiation exposure of the patient which opens up the possibility of increasing its application in a larger number of hospitals where CT is not acquired as a routine protocol. The proposed method consists of two main steps: First, the Mask R-CNN framework was employed to segment the pelvic shape from the background in the radiography images. Then, following the segmentation network, another convolutional network regressed the PSI angle. We employed a transfer learning paradigm where the network weights were initialized by non-medical images followed by fine-tuning using radiography images. Furthermore, in the training process, augmented data was generated to improve the performance of both networks. We analyzed the role of segmentation network in our system and investigated the Mask R-CNN performance in comparison with the U-Net, which is commonly used for the medical image segmentation. In this study, the Mask R-CNN utilizing multi-task learning, transfer learning, and data augmentation techniques achieve 0.960 ± 0.008 DICE coefficient, which significantly outperforms the U-Net. The cascaded system is capable of estimating the PSI with 4.04° ± 3.39° error for the radiography images. The proposed framework suggests a fully automatic and robust estimation of the PSI using only an anterior-posterior radiography image. [ABSTRACT FROM AUTHOR]

A rapid assessment of the seismic damage to buildings can facilitate improved emergency response and timely relief in earthquake-prone areas. In this study, an automated building seismic damage assessment method using an unmanned aerial vehicle (UAV) and a convolutional neural network (CNN) is introduced. The method consists of three parts: (1) data preparation, (2) building image segmentation, and (3) CNN-based building seismic damage assessment. First, a three-dimensional (3D) building model, aerial images, and camera data are used for the following simulation. Next, a building image segmentation method is proposed using the 3D building model as georeference, through which multi-view segmented building images can be obtained. Subsequently, a CNN model based on VGGNet is adopted to assess the seismic damage of each building. The CNN model is fine-tuned based on manually tagged building images obtained from the Internet. Finally, a case study of the old Beichuan town is used to demonstrate the effectiveness of the proposed method. The damage distribution of the area is obtained with an accuracy of 89.39%. • An automated seismic damage assessment framework is proposed for regional buildings. • A building identification method is proposed using 3D building models as georeference. • The CNN is adopted to assess the seismic damage of buildings based on aerial images. • The predicted building damage can be linked to the GIS data for risk management. [ABSTRACT FROM AUTHOR]

Liver and hepatic tumor segmentation remains a challenging problem in Computer Tomography (CT) images analysis due to its shape variation and vague boundary. The general hypothesis says that deep learning methods produce improved results on medical image segmentation. This paper formulates the segmentation of liver tumor in CT abdominal images as a classification problem, and then solves it using a cascaded classifier framework based on deep convolutional neural networks. Two deep encoder-decoder convolutional neural networks (EDCNN) were constructed and trained to cascade segments of both the liver and lesions in CT images with limited image quantity. In other words, an EDCNN segments the liver image as the input for the training of a second EDCNN. The second EDCNN then segments the tumor regions within the liver ROI regions as predicted by the first EDCNN. Segmenting the hepatic tumor inside the liver ROI also significantly reduces false-positives. The proposed model was then tested using a public dataset (3DIRCADb), and several metrics were used in order to quantitatively evaluate its performance. The proposed method produced an average DICE score of 95.22% for the test set of CT images. The proposed method was then compared with some of the existing methods. The experimental results demonstrated that the proposed EDCNN achieved improved performance in segmentation accuracy over some existing methods. [ABSTRACT FROM AUTHOR]

• We propose a new deep learning-based method for segmenting nasopharyngeal carcinoma (NPC) from axial, coronal and sagittal CT images. • The proposed method introduces a cascade strategy: (1) detecting and eliminating non-target organ regions from a given CT image. (2) Extracting NPC from the remained regions in the CT image. • The proposed method uses as its input several types of overlapping patches with different sizes. • From the experimental results using our database composed of 70 NPC patients, the proposed system archives the best performance for detecting NPC compared with conventional methods for detecting NPC directly from CT images. In the paper, we propose a new deep learning-based method for segmenting nasopharyngeal carcinoma (NPC) in the nasopharynx from three orthogonal CT images. The proposed method introduces a cascade strategy composed of two-phase manners. In CT images, there are organs, called non-target organs, which NPC never invades. Therefore, the first phase is to detect and eliminate non-target organ regions from the CT images. In the second phase, NPC is extracted from the remained regions in the CT images. Convolutional neural networks (CNNs) are applied to detect non-target organs and NPCs. The proposed system determines the final NPC segmentation by integrating three results obtained from coronal, axial and sagittal images. Moreover, we construct two CNN-based NPC detection systems using one kind of overlapping patches with a fixed size and various overlapping patches with different sizes. From the experiments using CT images of 70 NPC patients, our proposed systems, especially the system using various patches, achieves the best performance for detecting NPC compared with conventional NPC detection methods. [ABSTRACT FROM AUTHOR]

Cloud and cloud shadow segmentation is a crucial pre-processing step for any application that uses multi-spectral satellite images. In particular, disaster related applications (e.g., flood monitoring or rapid damage mapping), which are highly time- and data-critical, require methods that produce accurate cloud and cloud shadow masks in short time while being able to adapt to large variations in the target domain (induced by atmospheric conditions, different sensors, scene properties, etc.). In this study, we propose a data-driven approach to semantic segmentation of cloud and cloud shadow in single date images based on a modified U-Net convolutional neural network that aims to fulfil these requirements. We train the network on a global database of Landsat OLI images for the segmentation of five classes ("shadow", "cloud", "water", "land" and "snow/ice"). We compare the results to state-of-the-art methods, proof the model's generalization ability across multiple satellite sensors (Landsat TM, Landsat ETM+, Landsat OLI and Sentinel-2) and show the influence of different training strategies and spectral band combinations on the performance of the segmentation. Our method consistently outperforms Fmask and a traditional Random Forest classifier on a globally distributed multi-sensor test dataset in terms of accuracy, Cohen's Kappa coefficient, Dice coefficient and inference speed. The results indicate that a reduced feature space composed solely of red, green, blue and near-infrared bands already produces good results for all tested sensors. If available, adding shortwave-infrared bands can increase the accuracy. Contrast and brightness augmentations of the training data further improve the segmentation performance. The best performing U-Net model achieves an accuracy of 0.89, Kappa of 0.82 and Dice coefficient of 0.85, while running the inference over 896 test image tiles with 44.8 s/megapixel (2.8 s/megapixel on GPU). The Random Forest classifier reaches an accuracy of 0.79, Kappa of 0.65 and Dice coefficient of 0.74 with 3.9 s/megapixel inference time (on CPU) on the same training and testing data. The rule-based Fmask method takes significantly longer (277.8 s/megapixel) and produces results with an accuracy of 0.75, Kappa of 0.60 and Dice coefficient of 0.72. • Modified U-Net convolutional neural network to segment cloud, shadow, snow/ice, water and land in multi-spectral satellite images; • Focus on multi-sensor generalization ability, simplicity and computational efficiency; • Extensive performance evaluation across different sensors, seasons and locations; • Improved accuracy and inference speed compared to Fmask and Random Forest; • Shortwave-infrared bands and data augmentation can significantly improve the accuracy. [ABSTRACT FROM AUTHOR]