Search

As a biological signal existing in the human living body, the electrocardiogram (ECG) contains abundantly personal information and fulfils the basic characteristics of identity recognition. It has been widely used in the field of individual identification research in recent years. The common process of identity recognition includes three steps: ECG signals preprocessing, feature extraction and processing, beat classification recognition. However, the existing ECG classification models are sensitive to limitations of database type and extracted features dimension, which makes classification accuracy difficult to improve and cannot meet the needs of practical applications. To tackle the problem, this paper proposes to build an ECG individual recognition model based on a deep Long Short-Term Memory (LSTM) neural network. The LSTM network model has a memory cell and, therefore, it is an expert in handling long time ECG signals. With deeper learning, the nonlinear expression ability of the ECG beat classification model is gradually enhancing. The paper adopts two stacked LSTM models as hidden layers in the neural network; the Softmax layer is used as a classification layer to identify an individual. Then, low-level morphological features and deep-level chaotic features (Lyapunov exponent) are extracted to verify the feasibility of the deep LSTM network for classification. The model is respectively applied to a healthy human database and a human with a heart disease database. Experimental results show that extracting simple low-level features and chaotic features both achieve better classification performance. So, the robustness of the LSTM classification model is verified. [ABSTRACT FROM AUTHOR]

Palmprint identification has been conducted over the last two decades in many biometric systems. High-dimensional data with many uncorrelated and duplicated features remains difficult due to several computational complexity issues. This paper presents an interactive authentication approach based on deep learning and feature selection that supports Palmprint authentication. The proposed model has two stages of learning; the first stage is to transfer pre-trained VGG-16 of ImageNet to specific features based on the extraction model. The second stage involves the VGG-16 Palmprint feature extraction in the Siamese network to learn Palmprint similarity. The proposed model achieves robust and reliable end-to-end Palmprint authentication by extracting the convolutional features using VGG-16 Palmprint and the similarity of two input Palmprint using the Siamese network. The second stage uses the CASIA dataset to train and test the Siamese network. The suggested model outperforms comparable studies based on the deep learning approach achieving accuracy and EER of 91.8% and 0.082%, respectively, on the CASIA left-hand images and accuracy and EER of 91.7% and 0.084, respectively, on the CASIA right-hand images. [ABSTRACT FROM AUTHOR]

White blood cells (WBCs) constitute an essential part of the human immune system. The correct identification of WBC subtypes is critical in the diagnosis of leukemia, a kind of blood cancer defined by the aberrant proliferation of malignant leukocytes in the bone marrow. The traditional approach of classifying WBCs, which involves the visual analysis of blood smear images, is labor-intensive and error-prone. Modern approaches based on deep convolutional neural networks provide significant results for this type of image categorization, but have high processing and implementation costs owing to very large feature sets. This paper presents an improved hybrid approach for efficient WBC subtype classification. First, optimum deep features are extracted from enhanced and segmented WBC images using transfer learning on pre-trained deep neural networks, i.e., DenseNet201 and Darknet53. The serially fused feature vector is then filtered using an entropy-controlled marine predator algorithm (ECMPA). This nature-inspired meta-heuristic optimization algorithm selects the most dominant features while discarding the weak ones. The reduced feature vector is classified with multiple baseline classifiers with various kernel settings. The proposed methodology is validated on a public dataset of 5000 synthetic images that correspond to five different subtypes of WBCs. The system achieves an overall average accuracy of 99.9 % with more than 95 % reduction in the size of the feature vector. The feature selection algorithm also demonstrates better convergence performance as compared to classical meta-heuristic algorithms. The proposed method also demonstrates a comparable performance with several existing works on WBC classification. [ABSTRACT FROM AUTHOR]

This paper proposes a novel intelligent fault diagnosis method to automatically identify different health conditions of wind turbine (WT) gearbox. Unlike traditional approaches, where feature extraction and classification are separately designed and performed, this paper aims to automatically learn effective fault features directly from raw vibration signals while classify the type of faults in a single framework, thus providing an end-to-end learning-based fault diagnosis system for WT gearbox without additional signal processing and diagnostic expertise. Considering the multiscale characteristics inherent in vibration signals of a gearbox, a new multiscale convolutional neural network (MSCNN) architecture is proposed to perform multiscale feature extraction and classification simultaneously. The proposed MSCNN incorporates multiscale learning into the traditional CNN architecture, which has two merits: 1) high-level fault features can be effectively learned by the hierarchical learning structure with multiple pairs of convolutional and pooling layers; and 2) multiscale learning scheme can capture complementary and rich diagnosis information at different scales. This greatly improves the feature learning ability and enables better diagnosis performance. The proposed MSCNN approach is evaluated through experiments on a WT gearbox test rig. Experimental results and comprehensive comparison analysis with respect to the traditional CNN and traditional multiscale feature extractors have demonstrated the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

The reconstruction of urban buildings from large-scale airborne laser scanning point clouds is an important research topic in the geoscience field. Large-scale urban scenes usually contain a large number of object categories and many overlapped or closely neighboring objects, which poses great challenges for classifying and modeling buildings from these data sets. In this paper, we propose a deep reinforcement learning framework that integrates a 3-D convolutional neural network, a deep Q-network, and a residual recurrent neural network for the efficient semantic parsing of large-scale 3-D point clouds. The proposed framework provides an end-to-end automatic processing method that maps the raw point cloud to the classification results of the given categories. After obtaining the building classes, we utilize an edge-aware resampling algorithm to consolidate the point set with noise-free normals and clean preservation of sharp features. Finally, 2.5-D dual contouring, which is a data-driven approach, is introduced to generate urban building models from the consolidated point clouds. Our method can generate lightweight building models with arbitrarily shaped roofs while preserving the verticality of connecting walls. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) point cloud classification methods based on deep learning have good classification performance; however, they adapt poorly to diverse datasets and their classification accuracy must be improved. Therefore, FSDCNet, a neural network model based on the fusion of static and dynamic convolution, is proposed and applied for multiview 3D point cloud classification in this paper. FSDCNet devises a view selection method with fixed and random viewpoints, which effectively avoids the overfitting caused by the traditional fixed viewpoint. A local feature extraction operator of dynamic and static convolution adaptive weight fusion was designed to improve the model's adaptability to different types of datasets. To address the problems of large parameters and high computational complexity associated with the current methods of dynamic convolution, a lightweight and adaptive dynamic convolution operator was developed. In addition, FSDCNet builds a global attention pooling, integrating the most crucial information on different view features to the greatest extent. Due to these characteristics, FSDCNet is more adaptable, can extract more fine-grained detailed information, and can improve the classification accuracy of point cloud data. The proposed method was applied to the ModelNet40 and Sydney Urban Objects datasets. In these experiments, FSDCNet outperformed its counterparts, achieving state-of-the-art point cloud classification accuracy. For the ModelNet40 dataset, the overall accuracy (OA) and average accuracy (AA) of FSDCNet in a single view reached 93.8% and 91.2%, respectively, which were superior to those values for many other methods using 6 and 12 views. FSDCNet obtained the best results for 6 and 12 views, achieving 94.6%, 93.3%, 95.3%, and 93.6% in OA and AA metrics, respectively. For the Sydney Urban Objects dataset, FSDCNet achieved an OA and F1 score of 81.2% and 80.1% in a single view, respectively, which were higher than most of the compared methods. In 6 and 12 views, FSDCNet reached an OA of 85.3% and 83.6% and an F1 score of 85.5% and 83.7%, respectively. [ABSTRACT FROM AUTHOR]

Point cloud classification is one of the most important and time-consuming stages of airborne LiDAR (Light Detection and Ranging) data processing, playing a key role in the generation of cartographic products. This paper describes an innovative algorithm to perform LiDAR point-cloud classification, which relies on Convolutional Neural Networks (CNNs) and takes advantage of full-waveform data registered by modern laser scanners. The proposed method consists of two steps. First, a simple CNN is used to preprocess each waveform, providing a compact representation of the data. By exploiting the coordinates of the points associated with the waveforms, output vectors generated by the first CNN are then mapped into an image that is subsequently segmented by a Fully Convolutional Network (FCN): a label is assigned to each pixel and, consequently, to the point falling in the pixel. In this way, spatial positions and geometrical relationships between neighboring data are taken into account. These particular architectures allow to accurately identify even challenging classes such as power line and transmission tower. [ABSTRACT FROM AUTHOR]

This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervision. Instead, we use the same training data as traditional CNNs. Our method automatically assigns each interpretable filter in a high conv-layer with an object part of a certain category during the learning process. Such explicit knowledge representations in conv-layers of the CNN help people clarify the logic encoded in the CNN, i.e., answering what patterns the CNN extracts from an input image and uses for prediction. We have tested our method using different benchmark CNNs with various architectures to demonstrate the broad applicability of our method. Experiments have shown that our interpretable filters are much more semantically meaningful than traditional filters. [ABSTRACT FROM AUTHOR]

In semiconductor manufacturing, a wafer bin map (WBM) represents the results of wafer testing for dies using a binary pass or fail value. For WBMs, defective dies are often clustered into groups of local systematic defects. Determining their specific patterns is important, because different patterns are related to different root causes of failure. Recently, because wafer sizes have increased and the process technology has become more complicated, the probability of observing mixed-type defect patterns, i.e., two or more defect patterns in a single wafer, has increased. In this paper, we propose the use of convolutional neural networks (CNNs) to classify mixed-type defect patterns in WBMs in the framework of an individual classification model for each defect pattern. Through simulated and real data examples, we show that the CNN is robust to random noise and performs effectively, even if there are many random defects in WBMs. [ABSTRACT FROM AUTHOR]

For hyperspectral remote sensing image (HSI) classification, the learning process of deep neural networks has been progressively advanced in depth, but the fine features are often largely lost or even disappear in the process of depth transfer. With the increase in feature aggregation and connectivity, the complexity of the network and the training parameters increases greatly, requiring more training time. This paper proposed a multi-scale dense network (MSDN) for HSI classification that made full use of different scale information in the network structure and combined scale information throughout the network. It implemented feature extraction of HSIs in two dimensions, including the features at fine and coarse levels. In the horizontal direction, it considered the deep extraction of HSI features, and the 3-D dense connection structure was used for aggregating features at different levels. In the vertical direction, scale information was considered, and three-scale feature maps at low, middle, and high levels were generated based on the first layer of the network. The MSDN used stride convolution for downsampling and combined feature information at different scale levels. The MSDN extracted features along the diagonal line. The network implemented the reconstruction of deep feature extraction and multi-scale fusion for HSI classification. The MSDN model performed well on representative HSI datasets, namely, the Indian Pines, Pavia University, Salinas, Botswana, and Kennedy Space Center datasets. It improved the training speed and accuracy for HSI classification and especially improved the convergence speed, which effectively saved computing resources and had high stability. [ABSTRACT FROM AUTHOR]

In order to better interpret polarimetric synthetic aperture radar (PolSAR) images, many scholars tend to do target decomposition for PolSAR images and utilize the obtained features to perform subsequent classification. These target decomposition features play an important role in terrain classification but completely utilizing them produces a high computational complexity. Furthermore, some features have a negative impact on the classification task. Therefore, selecting the appropriate amount of high-quality features is of great significance to the classification task. In this paper, we propose a convolutional neural network (CNN)-based feature selection algorithm for PolSAR image classification. First, we design a 1-D CNN for feature selection, then train the designed network with all the decomposition features to obtain a trained model. Second, the Kullback–Leibler distance (KLD) between different features is utilized as a standard to select feature subsets. Third, feature subsets with excellent performance form the final results. Due to the special structure of the 1-D CNN, repetitively training model is avoided when the input changes. Different from traditional feature selection methods, our method considers the performance of features combination rather than single feature contribution. To this end, the feature subsets selected by the proposed method are more useful to the classification task. Innovatively introducing KLD in the selection stage avoids random selection and improves the selection efficiency. Finally, we validate the performance of selected feature subsets in traditional and deep learning classification frameworks. Experiments demonstrate that features selected by the proposed method have a good performance comparing with others on three real PolSAR data sets. [ABSTRACT FROM AUTHOR]

One in every four individuals has a diagnosable mental disease in a year. Around 20% of children and adolescents have a mental health condition and often ignore it. It is found that 93% of youth use social media to communicate and engage, as it reflects their emotions, moods, and thoughts. As a result, machine learning algorithms may anticipate people's moods and emotions based on their postings and comments. On the other hand, psychometric tests use questions to determine how individuals think, feel, behave, and react. It is necessary to investigate a hybrid approach for identifying people's mental illness by combining social media inputs and psychometric tests, especially during a pandemic. The hybrid approach can combine the results from both the models to reflect on the user's digital & non-digital reactions to certain sensitive situations to determine their mental state. Hence, the present paper aims to develop a web framework that can forecast the emergence of mental illness in the future based on data from social media comments and real-time data from psychometric tests using machine learning algorithms. The proposed work includes the cluster ensemble method for social media posts and a convolution neural network model for psychometric tests. This model predicts mental illness with an accuracy of 87.05 percent. The individual can use this result to take the required precautions by visiting a psychologist. [ABSTRACT FROM AUTHOR]

One of the challenges in motor imagery (MI) classification tasks is finding an easy-handled electroencephalogram (EEG) representation method which can preserve not only temporal features but also spatial ones. To fully utilize the features on various dimensions of EEG, a novel MI classification framework is first introduced in this paper, including a new 3D representation of EEG, a multi-branch 3D convolutional neural network (3D CNN) and the corresponding classification strategy. The 3D representation is generated by transforming EEG signals into a sequence of 2D array which preserves spatial distribution of sampling electrodes. The multi-branch 3D CNN and classification strategy are designed accordingly for the 3D representation. Experimental evaluation reveals that the proposed framework reaches state-of-the-art classification kappa value level and significantly outperforms other algorithms by 50% decrease in standard deviation of different subjects, which shows good performance and excellent robustness on different subjects. The framework also shows great performance with only nine sampling electrodes, which can significantly enhance its practicality. Moreover, the multi-branch structure exhibits its low latency and a strong ability in mitigating overfitting issues which often occur in MI classification because of the small training dataset. [ABSTRACT FROM AUTHOR]

The automatic classification of 3-D point clouds is publicly known as a challenging task in a complex road environment. Specifically, each point is automatically classified into a unique category label, and then, the labels are used as clues for semantic analysis and scene recognition. Instead of heuristically extracting handcrafted features in traditional methods to classify all points, we put forward an end-to-end octree-based fully convolutional network (FCN) to classify 3-D point clouds in an urban road environment. There are four contributions in this paper. The first is that the integration and comprehensive uses of OctNet and FCN greatly decrease the computing time and memory demands compared with a dense 3-D convolutional neural network (CNN). The second is that the octree-based network is strengthened by means of modifying the cross-entropy loss function to solve the problems of an unbalanced category distribution. The third is that an Inception-ResNet block is united with our network, which enables our 3-D CNN to effectively learn how to classify scenes containing objects at multiple scales and improve classification accuracy. The last is that an open source data set (HuangshiRoad data set) with ten different classes is introduced for 3-D point cloud classification. Three representative data sets [Semantic3D, WHU_MLS (blocks I and II), and HuangshiRoad] with different covered areas and numbers of points and classes are selected to evaluate our proposed method. The experimental results show that the overall classification accuracy is appreciable, with 89.4% for Semantic3D, 82.9% for WHU_MLS block I, 91.4% for WHU_MLS block II, and 94% for HuangshiRoad. Our deep learning approach can efficiently classify 3-D dense point clouds in an urban road environment measured by a mobile laser scanning (MLS) system or static LiDAR. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is a core task in the remote sensing community, and recently, deep learning-based methods have shown their capability of accurate classification of HSIs. Among the deep learning-based methods, deep convolutional neural networks (CNNs) have been widely used for the HSI classification. In order to obtain a good classification performance, substantial efforts are required to design a proper deep learning architecture. Furthermore, the manually designed architecture may not fit a specific data set very well. In this paper, the idea of automatic CNN for the HSI classification is proposed for the first time. First, a number of operations, including convolution, pooling, identity, and batch normalization, are selected. Then, a gradient descent-based search algorithm is used to effectively find the optimal deep architecture that is evaluated on the validation data set. After that, the best CNN architecture is selected as the model for the HSI classification. Specifically, the automatic 1-D Auto-CNN and 3-D Auto-CNN are used as spectral and spectral–spatial HSI classifiers, respectively. Furthermore, the cutout is introduced as a regularization technique for the HSI spectral–spatial classification to further improve the classification accuracy. The experiments on four widely used hyperspectral data sets (i.e., Salinas, Pavia University, Kennedy Space Center, and Indiana Pines) show that the automatically designed data-dependent CNNs obtain competitive classification accuracy compared with the state-of-the-art methods. In addition, the automatic design of the deep learning architecture opens a new window for future research, showing the huge potential of using neural architectures’ optimization capabilities for the accurate HSI classification. [ABSTRACT FROM AUTHOR]

The shortage of training samples remains one of the main obstacles in applying the neural networks to the hyperspectral images classification. To fuse the spatial and spectral information, pixel patches are often utilized to train a model, which may further aggregate this problem. In the existing works, an artificial neural network (ANN) model supervised by centerloss (ANNC) was introduced. Training merely with spectral information, the ANNC yields discriminative spectral features suitable for the subsequent classification tasks. In this paper, we propose a novel convolutional neural network (CNN)-based spatial feature fusion (CSFF) algorithm, which allows a smart integration of spatial information to the spectral features extracted by ANNC. As a critical part of CSFF, a CNN-based discriminant model is introduced to estimate whether two pixels belong to the same class. At the testing stage, by applying the discriminant model to the pixel pairs generated by a test pixel and each of its neighbors, the local structure is estimated and represented as a customized convolutional kernel. The spectral–spatial feature is generated by a convolutional operation between the estimated kernel and the corresponding spectral features within a local region. The final label is determined by classifying the resulting spectral–spatial feature. Without increasing the number of training samples or involving pixel patches at the training stage, the CSFF framework achieves the state of the art by declining 20%–50% classification failures in experiments on three well-known hyperspectral images. [ABSTRACT FROM AUTHOR]

Semantic labeling (or pixel-level land-cover classification) in ultrahigh-resolution imagery (<10 cm) requires statistical models able to learn high-level concepts from spatial data, with large appearance variations. Convolutional neural networks (CNNs) achieve this goal by learning discriminatively a hierarchy of representations of increasing abstraction. In this paper, we present a CNN-based system relying on a downsample-then-upsample architecture. Specifically, it first learns a rough spatial map of high-level representations by means of convolutions and then learns to upsample them back to the original resolution by deconvolutions. By doing so, the CNN learns to densely label every pixel at the original resolution of the image. This results in many advantages, including: 1) the state-of-the-art numerical accuracy; 2) the improved geometric accuracy of predictions; and 3) high efficiency at inference time. We test the proposed system on the Vaihingen and Potsdam subdecimeter resolution data sets, involving the semantic labeling of aerial images of 9- and 5-cm resolution, respectively. These data sets are composed by many large and fully annotated tiles, allowing an unbiased evaluation of models making use of spatial information. We do so by comparing two standard CNN architectures with the proposed one: standard patch classification, prediction of local label patches by employing only convolutions, and full patch labeling by employing deconvolutions. All the systems compare favorably or outperform a state-of-the-art baseline relying on superpixels and powerful appearance descriptors. The proposed full patch labeling CNN outperforms these models by a large margin, also showing a very appealing inference time. [ABSTRACT FROM PUBLISHER]

Deep-learning models play a significant role in modern software solutions, with the capabilities of handling complex tasks, improving accuracy, automating processes, and adapting to diverse domains, eventually contributing to advancements in various industries. This study provides a comparative study on deep-learning techniques that can also be deployed on resource-constrained edge devices. As a novel contribution, we analyze the performance of seven Convolutional Neural Network models in the context of data augmentation, feature extraction, and model compression using acoustic data. The results show that the best performers can achieve an optimal trade-off between model accuracy and size when compressed with weight and filter pruning followed by 8-bit quantization. In adherence to the study workflow utilizing the forest sound dataset, MobileNet-v3-small and ACDNet achieved accuracies of 87.95% and 85.64%, respectively, while maintaining compact sizes of 243 KB and 484 KB, respectively. Henceforth, this study concludes that CNNs can be optimized and compressed to be deployed in resource-constrained edge devices for classifying forest environment sounds. [ABSTRACT FROM AUTHOR]

Deep learning has been successfully applied to classification tasks in many fields due to its good performance in learning discriminative features. However, the application of deep learning to printing defect classification is very rare, and there is almost no research on the classification method for printing defects with imbalanced samples. In this paper, we present a deep convolutional neural network model to extract deep features directly from printed image defects. Furthermore, considering the asymmetry in the number of different types of defect samples—that is, the number of different kinds of defect samples is unbalanced—seven types of over-sampling methods were investigated to determine the best method. To verify the practical applications of the proposed deep model and the effectiveness of the extracted features, a large dataset of printing detect samples was built. All samples were collected from practical printing products in the factory. The dataset includes a coarse-grained dataset with four types of printing samples and a fine-grained dataset with eleven types of printing samples. The experimental results show that the proposed deep model achieves a 96.86% classification accuracy rate on the coarse-grained dataset without adopting over-sampling, which is the highest accuracy compared to the well-known deep models based on transfer learning. Moreover, by adopting the proposed deep model combined with the SVM-SMOTE over-sampling method, the accuracy rate is improved by more than 20% in the fine-grained dataset compared to the method without over-sampling. [ABSTRACT FROM AUTHOR]

Person re-identification is the task of matching pedestrian images across a network of non-overlapping camera views. It poses aggregated challenges resulted from random human pose, clutter from the background, illumination variations, and other factors. There has been a vast number of studies in recent years with promising success. However, key challenges have not been adequately addressed and continue to result in sub-optimal performance. Attention-based person re-identification gains more popularity in identifying discriminatory features from person images. Its potential in terms of extracting features common to a pair of person images across the feature extraction pipeline has not been be fully exploited. In this paper, we propose a novel attention-based Siamese network driven by a mutual-attention module decomposed into spatial and channel components. The proposed mutual-attention module not only leads feature extraction to the discriminative part of individual images, but also fuses mutual features symmetrically across pairs of person images to get informative regions common to both input images. Our model simultaneously learns feature embedding for discriminative cues and the similarity measure. The proposed model is optimized with multi-task loss, namely classification and verification loss. It is further optimized by a learnable mutual-attention module to facilitate an efficient and adaptive learning. The proposed model is thoroughly evaluated on extensively used large-scale datasets, Market-1501 and Duke-MTMC-ReID. Our experimental results show competitive results with the state-of-the-art works and the effectiveness of the mutual-attention module. [ABSTRACT FROM AUTHOR]

The convolutional neural network (CNN) has been gradually applied to the hyperspectral images (HSIs) classification, but the lack of training samples caused by the difficulty of HSIs sample marking and ignoring of correlation between spatial and spectral information seriously restrict the HSIs classification accuracy. In an attempt to solve these problems, this paper proposes a dual-branch extraction and classification method under limited samples of hyperspectral images based on deep learning (DBECM). At first, a sample augmentation method based on local and global constraints in this model is designed to augment the limited training samples and balance the number of different class samples. Then spatial-spectral features are simultaneously extracted by the dual-branch spatial-spectral feature extraction method, which improves the utilization of HSIs data information. Finally, the extracted spatial-spectral feature fusion and classification are integrated into a unified network. The experimental results of two typical datasets show that the DBECM proposed in this paper has certain competitive advantages in classification accuracy compared with other public HSIs classification methods, especially in the Indian pines dataset. The parameters of the overall accuracy (OA), average accuracy (AA), and Kappa of the method proposed in this paper are at least 4.7%, 5.7%, and 5% higher than the existing methods. [ABSTRACT FROM AUTHOR]

3D convolutional neural networks (CNNs) have been demonstrated to be a powerful tool in hyperspectral images (HSIs) classification. However, using the conventional 3D CNNs to extract the spectral–spatial feature for HSIs results in too many parameters as HSIs have plenty of spatial redundancy. To address this issue, in this paper, we first design multiscale convolution to extract the contextual feature of different scales for HSIs and then propose to employ the octave 3D CNN which factorizes the mixed feature maps by their frequency to replace the normal 3D CNN in order to reduce the spatial redundancy and enlarge the receptive field. To further explore the discriminative features, a channel attention module and a spatial attention module are adopted to optimize the feature maps and improve the classification performance. The experiments on four hyperspectral image data sets demonstrate that the proposed method outperforms other state-of-the-art deep learning methods. [ABSTRACT FROM AUTHOR]

In recent years, Peripheral blood smear is a generic analysis to assess the person’s health status. Manual testing of Peripheral blood smear images are difficult, time-consuming and is subject to human intervention and visual error. This method encouraged for researchers to present algorithms and techniques to perform the peripheral blood smear analysis with the help of computer-assisted and decision-making techniques. Existing CAD based methods are lacks in attaining the accurate detection of abnormalities present in the images. In order to mitigate this issue Deep Convolution Neural Network (DCNN) based automatic classification technique is introduced with the classification of eight groups of peripheral blood cells such as basophil, eosinophil, lymphocyte, monocyte, neutrophil, erythroblast, platelet, myocyte, promyocyte and metamyocyte. The proposed DCNN model employs transfer learning approach and additionally it carries three stages such as pre-processing, feature extraction and classification. Initially the pre-processing steps are incorporated to eliminate noisy contents present in the image by using Histogram Equalization (HE). It is enclosed to improve an image contrast. In order to distinguish the dissimilar class and segmentation approach is carried out with the help of Fuzzy C-Means (FCM) model whereas its centroid point optimality method with Slap Swarm based optimization strategy. Moreover some specific set of Gray Level Co-occurrence Matrix (GLCM) features of the segmented images are extracted to augment the performance of proposed detection algorithm. Finally the extracted features are recorded by DCNN and the proposed classifier has the capability to extract their own features. Based on this the diverse set of classes are classified and distinguished from qualitative abnormalities found in the image. [ABSTRACT FROM AUTHOR]

Since the existing music emotion classification researches focus on the single-modal analysis of audio or lyrics, the correlation among models are neglected, which lead to partial information loss. Therefore, a music emotion classification method based on deep learning and improved attention mechanism is proposed. First, the music lyrics features are extracted by Term Frequency-Inverse Document Frequency (TF-IDF) and Word2vec method, and the term frequency weight vector and word vector are obtained. Then, by using the feature extraction ability of Convolutional Neural Network (CNN) and the ability of Long Short-Term Memory (LSTM) network to process the serialized data, and integrating the matching attention mechanism, an emotion analysis model based on CNN-LSTM is constructed. Finally, the output results of the deep neural network and CNN-LSTM model are fused, and the emotion types are obtained by Softmax classifier. The experimental analysis based on the selected data sets shows that the average classification accuracy of the proposed method is 0.848, which is better than the other comparison methods, and the classification efficiency has been greatly improved. [ABSTRACT FROM AUTHOR]

The lack of labels is one of the major challenges in hyperspectral image (HSI) classification. Widely used Deep Learning (DL) models such as convolutional neural networks (CNNs) experience serious performance degradation when training samples are limited. In contrast, graph convolutional networks (GCNs) can simultaneously exploit the insufficient labeled data and massive unlabeled data of HSI in a semisupervised learning fashion. However, in order to reduce computational cost and mitigate noise, existing GCN-based classification methods usually perform superpixel segmentation as a preprocessing step and implement feature extraction as well as node classification on the predefined superpixel graph, where one superpixel might incorporate pixels with different labels. Moreover, the local spectral–spatial information within superpixels is generally ignored. To alleviate these two issues, we propose an end-to-end mixhop superpixel-based GCN (EMS-GCN) framework for HSI classification. Specifically, we first introduce the differentiable superpixel segmentation algorithm to map the pixel representations into a superpixel feature space, which allows refining the superpixel boundary with the training of the network. After that, a superpixel graph is constructed and fed into a novel mixhop superpixel-based GCN, where both the local information within superpixels and long-range information among superpixels are extracted, while the structure of the superpixel graph is updated at the same time. Finally, the enhanced superpixel representations are mapped back into a pixel feature space to conduct pixel-wise classification. Extensive experiments demonstrate the effectiveness of the proposed EMS-GCN method compared with other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Uncertainty modeling of renewable energy sources, load demand, electricity price, etc. create a high volume of data in smart grids. Accordingly, in this article, a precise forecasting method based on a deep learning concept with microclustering (MC) task is presented. The MC method is structured based on hybrid unsupervised and supervised clustering tasks by K-means and Gaussian support vector machine, respectively. In the proposed method, the input data sequence is clustered by the MC task, and then the forecasting process is employed. By applying the MC, input data in each hour are categorized into different groups, and a distinctive forecasting unit is allocated to each one. In this way, more clusters and forecasting networks are earmarked for the hours with higher fluctuation rates. The bi-directional long short-term memory (B-LSTM), which is one of the newest recurrent artificial neural networks, is proposed as the forecasting unit. The B-LSTM has bidirectional memory—feedforward and feedback loops—that helps us to investigate both previous and future hidden layers data. The optimal number of clusters in each hour is determined based on the Davies–Bouldin index. To evaluate the performance of the proposed method, in this study, three forecasting tasks including the wind speed, load demand, and electricity price are studied in different periods using the Ontario province, Canada, data set. The results are compared with other benchmarking methods to verify the robustness and effectiveness of the proposed method. In fact, the proposed method, which is equipped with the MC technique and B-LSTM networks, significantly promotes the forecasting results, especially in spike points. [ABSTRACT FROM AUTHOR]

Classification and identification of the materials lying over or beneath the earth’s surface have long been a fundamental but challenging research topic in geoscience and remote sensing (RS), and have garnered a growing concern owing to the recent advancements of deep learning techniques. Although deep networks have been successfully applied in single-modality-dominated classification tasks, yet their performance inevitably meets the bottleneck in complex scenes that need to be finely classified, due to the limitation of information diversity. In this work, we provide a baseline solution to the aforementioned difficulty by developing a general multimodal deep learning (MDL) framework. In particular, we also investigate a special case of multi-modality learning (MML)—cross-modality learning (CML) that exists widely in RS image classification applications. By focusing on “what,” “where,” and “how” to fuse, we show different fusion strategies as well as how to train deep networks and build the network architecture. Specifically, five fusion architectures are introduced and developed, further being unified in our MDL framework. More significantly, our framework is not only limited to pixel-wise classification tasks but also applicable to spatial information modeling with convolutional neural networks (CNNs). To validate the effectiveness and superiority of the MDL framework, extensive experiments related to the settings of MML and CML are conducted on two different multimodal RS data sets. Furthermore, the codes and data sets will be available at https://github.com/danfenghong/IEEE%5fTGRS%5fMDL-RS , contributing to the RS community. [ABSTRACT FROM AUTHOR]

Recently, multimodal cloud samples are utilized to learn completed feature representations for cloud classification. However, the existing methods neglect the related information from other multimodal cloud samples in the learning process, which leads to inadequate learning. In this article, we propose a novel deep model to learn heterogeneous deep features (HDFs) for multimodal ground-based remote sensing cloud classification. Specifically, we first design the convolutional neural network (CNN) extractor to combine the visual information and the multimodal information (MI) to obtain the CNN-based features of multimodal cloud samples. Afterward, we treat the CNN-based features of multimodal cloud samples as the nodes of graph, and utilize the similarity between nodes as the adjacency matrix. We feed the graph and the adjacency matrix into the graph convolutional network (GCN) extractor to obtain the GCN-based features that could capture correlations among multimodal cloud samples using graph convolutional layers. After obtaining CNN-based features and GCN-based features, we concatenate the two kinds of heterogeneous features to represent the multimodal cloud samples. As a result, the concatenated feature contains the visual information, the MI and the related information among multimodal cloud samples. We conduct a series of experiments on the multimodal ground-based cloud database (MGCD), and the experimental results verify that the proposed HDF outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Huge amount of time series data over several domains such as engineering, biomedical and finance, demands the development of efficient methods for the problem of time series classification. The classification of univariate and multivariate time series together using a single architecture is a very difficult task. In this work, a bio-inspired convolutional spiking neural network (CSNN) is proposed for both univariate and multivariate time series. For this, first we develop a simple transformation to convert raw time series sequences into matrices. The CSNN is a three staged framework which include convolutional feature extraction, spike encoding using soft leaky integrate and fire (Soft-LIf) and classification. As spikes generated are differentiable, thus the learning algorithm for CSNN uses error-backpropagation with cyclical learning rates (CLR) and RMSprop optimizer. Additionally, validation based stopping rules are employed to overcome the overfitting which also provides a set of parameters associated with low validation set loss. Thereafter, to demonstrate the accuracy and robustness of proposed CSNN model, we have used University of California (UCR) univariate as well as University of East Anglia (UEA) multivariate datasets to perform the experiments. Moreover, we conduct comparative empirical performance evaluation with benchmark methods and also with recent deep networks proposed for time series classification. Our results reveal that proposed CSNN advances the baseline methods by achieving higher performance accuracy for both univariate and multivariate datasets. It is shown that the CLR with RMSprop optimizer is able to achieve faster convergence, however CLR and adaptive rates are considered competitive to each other. In addition, we also address the optimal model selection and study the effects of different factors on the performance of proposed CSNN. [ABSTRACT FROM AUTHOR]

Deep learning models have been successfully applied to the analysis of various functional MRI data. Convolutional neural networks (CNN), a class of deep neural networks, have been found to excel at extracting local meaningful features based on their shared-weights architecture and space invariance characteristics. In this study, we propose M2D CNN, a novel multichannel 2D CNN model, to classify 3D fMRI data. The model uses sliced 2D fMRI data as input and integrates multichannel information learned from 2D CNN networks. We experimentally compared the proposed M2D CNN against several widely used models including SVM, 1D CNN, 2D CNN, 3D CNN, and 3D separable CNN with respect to their performance in classifying task-based fMRI data. We tested M2D CNN against six models as benchmarks to classify a large number of time-series whole-brain imaging data based on a motor task in the Human Connectome Project (HCP). The results of our experiments demonstrate the following: (i) convolution operations in the CNN models are advantageous for high-dimensional whole-brain imaging data classification, as all CNN models outperform SVM; (ii) 3D CNN models achieve higher accuracy than 2D CNN and 1D CNN model, but 3D CNN models are computationally costly as any extra dimension is added in the input; (iii) the M2D CNN model proposed in this study achieves the highest accuracy and alleviates data overfitting given its smaller number of parameters as compared with 3D CNN. [ABSTRACT FROM AUTHOR]

Ground-penetrating radar (GPR) target detection and classification is a challenging task. Here, we consider online dictionary learning (DL) methods to obtain sparse representations (SR) of the GPR data to enhance feature extraction for target classification via support vector machines. Online methods are preferred because traditional batch DL like K-times singular value decomposition (K-SVD) is not scalable to high-dimensional training sets and infeasible for real-time operation. We also develop Drop-Off MINi-batch Online Dictionary Learning (DOMINODL), which exploits the fact that a lot of the training data may be correlated. The DOMINODL algorithm iteratively considers elements of the training set in small batches and drops off samples which become less relevant. For the case of abandoned anti-personnel landmines classification, we compare the performance of K-SVD with three online algorithms: classical online dictionary learning (ODL), its correlation-based variant, and DOMINODL. Our experiments with real data from L-band GPR show that online DL methods reduce learning time by 36%–93% and increase mine detection by 4%–28% over K-SVD. Our DOMINODL is the fastest and retains similar classification performance as the other two online DL approaches. We use a Kolmogorov–Smirnoff test distance and the Dvoretzky–Kiefer–Wolfowitz inequality for the selection of DL input parameters leading to enhanced classification results. To further compare with the state-of-the-art classification approaches, we evaluate a convolutional neural network (CNN) classifier, which performs worse than the proposed approach. Moreover, when the acquired samples are randomly reduced by 25%, 50%, and 75%, sparse decomposition-based classification with DL remains robust while the CNN accuracy is drastically compromised. [ABSTRACT FROM AUTHOR]

This study aims to propose a vision-based method to classify mosquito species. To investigate the efficiency of the method, we compared two different classification methods: The handcraft feature-based conventional method and the convolutional neural network-based deep learning method. For the conventional method, 12 types of features were adopted for handcraft feature extraction, while a support vector machine method was adopted for classification. For the deep learning method, three types of architectures were adopted for classification. We built a mosquito image dataset, which included 14,400 images with three types of mosquito species. The dataset comprised 12,000 images for training, 1500 images for testing, and 900 images for validating. Experimental results revealed that the accuracy of the conventional method using the scale-invariant feature transform algorithm was 82.4% at maximum, whereas the accuracy of the deep learning method was 95.5% in a residual network using data augmentation. From the experimental results, deep learning can be considered to be effective for classifying the mosquito species of the proposed dataset. Furthermore, data augmentation improves the accuracy of mosquito species' classification. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have strong feature extraction capability, which have been used to extract features from the hyperspectral image. Local binary pattern (LBP) is a simple but powerful descriptor for spatial features, which can lessen the workload of CNNs and improve the classification accuracy. In order to make full use of the feature extraction capability of CNNs and the discrimination of LBP features, a novel classification method combining dual-channel CNNs and LBP is proposed. Specifically, a one-dimensional CNN (1D-CNN) is adopted to process original hyperspectral data to extract hierarchical spectral features and another same 1D-CNN is applied to process LBP features to further extract spatial features. Then, the concatenation of two fully connected layers from the two CNNs, which fused features, is fed into a softmax classifier to complete the classification. The experimental results demonstrate that the proposed method can provide 98.52%, 99.54% and 99.54% classification accuracy on the Indian Pines, University of Pavia and Salinas data, respectively. And the proposed method can also obtain good performance even with limited training samples. [ABSTRACT FROM AUTHOR]

Most traditional approaches classify hyperspectral image (HSI) pixels relying only on the spectral values of the input channels. However, the spatial context around a pixel is also very important and can enhance the classification performance. In order to effectively exploit and fuse both the spatial context and spectral structure, we propose a novel two-stream deep architecture for HSI classification. The proposed method consists of a two-stream architecture and a novel fusion scheme. In the two-stream architecture, one stream employs the stacked denoising autoencoder to encode the spectral values of each input pixel, and the other stream takes as input the corresponding image patch and deep convolutional neural networks are employed to process the image patch. In the fusion scheme, the prediction probabilities from two streams are fused by adaptive class-specific weights, which can be obtained by a fully connected layer. Finally, a weight regularizer is added to the loss function to alleviate the overfitting of the class-specific fusion weights. Experimental results on real HSIs demonstrate that the proposed two-stream deep architecture can achieve competitive performance compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, the development of convolutional neural networks (CNNs) has promoted continuous progress in scene classification of remote sensing images. Compared with natural image datasets, however, the acquisition of remote sensing scene images is more difficult, and consequently the scale of remote sensing image datasets is generally small. In addition, many problems related to small objects and complex backgrounds arise in remote sensing image scenes, presenting great challenges for CNN-based recognition methods. In this article, to improve the feature extraction ability and generalization ability of such models and to enable better use of the information contained in the original remote sensing images, we introduce a multitask learning framework which combines the tasks of self-supervised learning and scene classification. Unlike previous multitask methods, we adopt a new mixup loss strategy to combine the two tasks with dynamic weight. The proposed multitask learning framework empowers a deep neural network to learn more discriminative features without increasing the amounts of parameters. Comprehensive experiments were conducted on four representative remote sensing scene classification datasets. We achieved state-of-the-art performance, with average accuracies of 94.21%, 96.89%, 99.11%, and 98.98% on the NWPU, AID, UC Merced, and WHU-RS19 datasets, respectively. The experimental results and visualizations show that our proposed method can learn more discriminative features and simultaneously encode orientation information while effectively improving the accuracy of remote sensing scene classification. [ABSTRACT FROM AUTHOR]

Every pixel in a hyperspectral image contains detailed spectral information in hundreds of narrow bands captured by hyperspectral sensors. Pixel-wise classification of a hyperspectral image is the cornerstone of various hyperspectral applications. Nowadays, deep learning models represented by the convolutional neural network (CNN) provides an ideal solution for feature extraction, and has made remarkable achievements in supervised hyperspectral classification. However, hyperspectral image annotation is time-consuming and laborious, and available training data is usually limited. Due to the "small-sample problem", CNN-based hyperspectral classification is still challenging. Focused on the limited sample-based hyperspectral classification, we designed an 11-layer CNN model called R-HybridSN (Residual-HybridSN) from the perspective of network optimization. With an organic combination of 3D-2D-CNN, residual learning, and depth-separable convolutions, R-HybridSN can better learn deep hierarchical spatial–spectral features with very few training data. The performance of R-HybridSN is evaluated over three public available hyperspectral datasets on different amounts of training samples. Using only 5%, 1%, and 1% labeled data for training in Indian Pines, Salinas, and University of Pavia, respectively, the classification accuracy of R-HybridSN is 96.46%, 98.25%, 96.59%, respectively, which is far better than the contrast models. [ABSTRACT FROM AUTHOR]

Presently, most deaths are caused by heart disease. To overcome this situation, heartbeat sound analysis is a convenient way to diagnose heart disease. Heartbeat sound classification is still a challenging problem in heart sound segmentation and feature extraction. Dataset-B applied in this study that contains three categories Normal, Murmur and Extra-systole heartbeat sound. In the purposed framework, we remove the noise from the heartbeat sound signal by applying the band filter, After that we fixed the size of the sampling rate of each sound signal. Then we applied down-sampling techniques to get more discriminant features and reduce the dimension of the frame rate. However, it does not affect the results and also decreases the computational power and time. Then we applied a purposed model Recurrent Neural Network (RNN) that is based on Long Short-Term Memory (LSTM), Dropout, Dense and Softmax layer. As a result, the purposed method is more competitive compared to other methods. [ABSTRACT FROM AUTHOR]

The use of Convolutional Neural Networks (CNNs) to solve Domain Adaptation (DA) image classification problems in the context of remote sensing has proven to provide good results but at high computational cost. To avoid this problem, a deep learning network for DA in remote sensing hyperspectral images called TCANet is proposed. As a standard CNN, TCANet consists of several stages built based on convolutional filters that operate on patches of the hyperspectral image. Unlike the former, the coefficients of the filter are obtained through Transfer Component Analysis (TCA). This approach has two advantages: firstly, TCANet does not require training based on backpropagation, since TCA is itself a learning method that obtains the filter coefficients directly from the input data. Second, DA is performed on the fly since TCA, in addition to performing dimensional reduction, obtains components that minimize the difference in distributions of data in the different domains corresponding to the source and target images. To build an operating scheme, TCANet includes an initial stage that exploits the spatial information by providing patches around each sample as input data to the network. An output stage performing feature extraction that introduces sufficient invariance and robustness in the final features is also included. Since TCA is sensitive to normalization, to reduce the difference between source and target domains, a previous unsupervised domain shift minimization algorithm consisting of applying conditional correlation alignment (CCA) is conditionally applied. The results of a classification scheme based on CCA and TCANet show that the DA technique proposed outperforms other more complex DA techniques. [ABSTRACT FROM AUTHOR]