Search

Recent technological developments have enabled computers to identify and categorize facial expressions to determine a person's emotional state in an image or a video. This process, called "Facial Expression Recognition (FER)", has become one of the most popular research areas in computer vision. In recent times, deep FER systems have primarily concentrated on addressing two significant challenges: the problem of overfitting due to limited training data availability, and the presence of expression-unrelated variations, including illumination, head pose, image resolution, and identity bias. In this paper, a comprehensive survey is provided on deep FER, encompassing algorithms and datasets that offer insights into these intrinsic problems. Initially, this paper presents a detailed timeline showcasing the evolution of methods and datasets in deep facial expression recognition (FER). This timeline illustrates the progression and development of the techniques and data resources used in FER. Then, a comprehensive review of FER methods is introduced, including the basic principles of FER (components such as preprocessing, feature extraction and classification, and methods, etc.) from the pro-deep learning era (traditional methods using handcrafted features, i.e., SVM and HOG, etc.) to the deep learning era. Moreover, a brief introduction is provided related to the benchmark datasets (there are two categories: controlled environments (lab) and uncontrolled environments (in the wild)) used to evaluate different FER methods and a comparison of different FER models. Existing deep neural networks and related training strategies designed for FER, based on static images and dynamic image sequences, are discussed. The remaining challenges and corresponding opportunities in FER and the future directions for designing robust deep FER systems are also pinpointed. [ABSTRACT FROM AUTHOR]

Power quality improvement and Power quality disturbance (PQD) detection are two significant concerns that must be addressed to ensure an efficient power distribution within the utility grid. When the process to analyze PQD is migrated to real-time platforms, the possible occurrence of a phase mismatch can affect the algorithm's accuracy; this paper evaluates phase shifting as an additional stage in signal acquisition for detecting and classifying eight types of single power quality disturbances. According to their mathematical models, a set of disturbances was generated using an arbitrary waveform generator BK Precision 4064. The acquisition, detection, and classification stages were embedded into a BeagleBone Black. The detection stage was performed using multiresolution analysis. The feature vectors of the acquired signals were obtained from the combination of Shannon entropy and log-energy entropy. For classification purposes, four types of classifiers were trained: multilayer perceptron, K-nearest neighbors, probabilistic neural network, and decision tree. The results show that incorporating a phase-shifting stage as a preprocessing stage significantly improves the classification accuracy in all cases. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs' spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs. [ABSTRACT FROM AUTHOR]

The accurate semantic segmentation of point cloud data is the basis for their application in the inspection of extra high-voltage transmission lines (EHVTL). As deep learning evolves, point-wise-based deep neural networks have shown great potential for the semantic segmentation of EHVTL point clouds. However, EHVTL point cloud data are characterized by a large data volume and significant class imbalance. Therefore, the down-sampling method and point cloud feature extraction method used in current point-wise-based deep neural networks hardly meet the needs of computational accuracy and efficiency. In this paper, we proposed a two-step down-sampling method and a point cloud feature extraction method based on local feature aggregation of the point clouds after down-sampling in each layer of the model (LFAPAD). We then established a deep neural network named PowerLine-Net for the semantic segmentation of the EHVTL point clouds. Furthermore, in order to test and analyze the performance of PowerLine-Net, we constructed a point cloud dataset for the EHVTL scenes. Using this dataset and the Semantic3D dataset, we implemented network parameter testing, semantic segmentation, and an accuracy comparison of different networks based on PowerLine-Net. The results illustrate that the semantic segmentation model proposed in this paper has a high computational efficiency and accuracy in the semantic segmentation of EHVTL point clouds. Compared with conventional deep neural networks, including PointCNN, KPConv, SPG, PointNet++, and RandLA-Net, PowerLine-Net also achieves a higher accuracy in the semantic segmentation of EHVTL point clouds. Moreover, based on the results predicted by PowerLine-Net, the risk point detection for EHVTL point clouds has been achieved, which demonstrates the important value of this network in practical applications. In addition, as shown by the results of Semantic3D, PowerLine-Net also achieves a high segmentation accuracy, which proves its powerful capability and wide applicability in semantic segmentation for the point clouds of large-scale scenes. [ABSTRACT FROM AUTHOR]

Marine oil spills can cause serious damage to marine ecosystems and biological species, and the pollution is difficult to repair in the short term. Accurate oil type identification and oil thickness quantification are of great significance for marine oil spill emergency response and damage assessment. In recent years, hyperspectral remote sensing technology has become an effective means to monitor marine oil spills. The spectral and spatial features of oil spill images at different levels are different. To accurately identify oil spill types and quantify oil film thickness, and perform better extraction of spectral and spatial features, a multilevel spatial and spectral feature extraction network is proposed in this study. First, the graph convolutional neural network and graph attentional neural network models were used to extract spectral and spatial features in non-Euclidean space, respectively, and then the designed modules based on 2D expansion convolution, depth convolution, and point convolution were applied to extract feature information in Euclidean space; after that, a multilevel feature fusion method was developed to fuse the obtained spatial and spectral features in Euclidean space in a complementary way to obtain multilevel features. Finally, the multilevel features were fused at the feature level to obtain the oil spill information. The experimental results show that compared with CGCNN, SSRN, and A2S2KResNet algorithms, the accuracy of oil type identification and oil film thickness classification of the proposed method in this paper is improved by 12.82%, 0.06%, and 0.08% and 2.23%, 0.69%, and 0.47%, respectively, which proves that the method in this paper can effectively extract oil spill information and identify different oil spill types and different oil film thicknesses. [ABSTRACT FROM AUTHOR]

In this paper, laser-induced breakdown spectroscopy (LIBS) combined with a probabilistic neural network (PNN) was applied to classify engineering structural metal samples (valve stem, welding material, and base metal). Additionally, utilizing data from the plasma emission spectrum generated by laser ablation of samples with different aging times, an aging time prediction model based on a firefly optimized probabilistic neural network (FA-PNN) was established, which can effectively evaluate the service performance of structural materials. The problem of insufficient features obtained by principal component analysis (PCA) for predicting the aging time of materials is addressed by the proposal of a time-frequency feature extraction method based on short-time Fourier transform (STFT). The classification accuracy (ACC) of time-frequency features and principal component features was compared under PNN. The results indicate that, in comparison to the PCA feature extraction approach, the time-frequency feature extraction method based on STFT demonstrates higher accuracy in predicting the time of aging materials. Then, the relationship between classification accuracy (ACC) and settings of PNN was discussed. The ACC of the PNN model for both the material classification test set and the aging time test set achieved 100% with Firefly (FA) optimization algorithms. This result was also compared with the ACC of ANN, KNN, PLS-DA, and SIMCA for the aging time test set (95%, 87.5%, 85%, and 62.5%, respectively). The experimental results demonstrated that the classification model using LIBS combined with FA-PNN could realize better classification accuracy. [ABSTRACT FROM AUTHOR]

Blade icing seriously affects wind turbines' aerodynamic performance and output power. Timely and accurately predicting blade icing status is crucial to improving the economy and safety of wind farms. However, existing blade icing prediction methods cannot effectively solve the problems of unbalanced icing/non-icing data and low prediction accuracy. In order to solve the above problems, this paper proposes a wind turbine blade icing prediction method based on the focal loss function and CNN-Attention-GRU. First, the recursive feature elimination method combined with the physical mechanism of icing is used to extract features highly correlated with blade icing, and a new feature subset is formed through a sliding window algorithm. Then, the focal loss function is utilized to assign more weight to the ice samples with a lower proportion, addressing the significant class imbalance between the ice and non-ice categories. Finally, based on the CNN-Attention-GRU algorithm, a blade icing prediction model is established using continuous 24-h historical data as the input and the icing status of the next 24 h as the output. The model is compared with advanced neural network models. The results show that the proposed method improves the prediction accuracy and F1 score by an average of 6.41% and 4.27%, respectively, demonstrating the accuracy and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In recent years, data-driven deep networks have demonstrated remarkable detection performance for infrared small targets. However, continuously increasing the depth of neural networks to enhance performance has proven impractical. Consequently, the integration of prior physical knowledge related to infrared small targets within deep neural networks has become crucial. It aims to improve the models' awareness of inherent physical characteristics. In this paper, we propose a novel dual-domain prior-driven deep network (DPDNet) for infrared small-target detection. Our method integrates the advantages of both data-driven and model-driven methods by leveraging the prior physical characteristics as the driving force. Initially, we utilize the sparse characteristics of infrared small targets to boost their saliency at the input level of the network. Subsequently, a high-frequency feature extraction module, seamlessly integrated into the network's backbone, is employed to excavate feature information. DPDNet simultaneously emphasizes the prior sparse characteristics of infrared small targets in the spatial domain and their prior high-frequency characteristics in the frequency domain. Compared with previous CNN-based methods, our method achieves superior performance while utilizing fewer convolutional layers. It has a performance of 78.64% IoU, 95.56 Pd, and 2.15 × 10−6 Fa on the SIRST dataset. [ABSTRACT FROM AUTHOR]

Speech separation is a well-known problem, especially when there is only one sound mixture available. Estimating the Ideal Binary Mask (IBM) is one solution to this problem. Recent research has focused on the supervised classification approach. The challenge of extracting features from the sources is critical for this method. Speech separation has been accomplished by using a variety of feature extraction models. The majority of them, however, are concentrated on a single feature. The complementary nature of various features have not been thoroughly investigated. In this paper, we propose a deep neural network (DNN) ensemble architecture to completely explore the complimentary nature of the diverse features obtained from raw acoustic features. We examined the penultimate discriminative representations instead of employing the features acquired from the output layer. The learned representations were also fused to produce a new features vector, which was then classified by using the Extreme Learning Machine (ELM). In addition, a genetic algorithm (GA) was created to optimize the parameters globally. The results of the experiments showed that our proposed system completely considered various features and produced a high-quality IBM under different conditions. [ABSTRACT FROM AUTHOR]

Image recognition has long been one of the research hotspots in computer vision tasks. The development of deep learning is rapid in recent years, and convolutional neural networks usually need to be designed with fixed resources. If sufficient resources are available, the model can be scaled up to achieve higher accuracy, for example, VggNet, ResNet, GoogLeNet, etc. Although the accuracy of large-scale models has been improved, the following problems will occur with the expansion of model scale: (1) There may be over-fitting; (2) increasing model parameters; (3) slow model convergence. This paper proposes a design method for a modular convolutional neural network model which solves the problem of over-fitting and large model parameters by connecting multiple modules in parallel. Moreover, each module contains several submodules (three submodules in this paper) and fuses the features extracted from the submodules. The model convergence can be accelerated by using the fused features (the fused features contain more image information). In this study, we add a gate unit based on the attention mechanism to the model, which aims to optimize the structure of the model (select the optimal number of modules), allowing the model to select an optimum network structure by learning and dynamically reducing FLOPs (floating-point operations per second) of the model. Compared to VggNet, ResNet, and GoogLeNet, the structure of the model proposed in this paper is simple and the parameters are small. The proposed model achieves good results in the Kaggle datasets Cats-vs.-Dogs (99.3%), 10-Monkey Species (99.26%), and Birds-400 (99.13%). [ABSTRACT FROM AUTHOR]

Simple Summary: Breast cancer is one of the leading causes of cancer death among women. Developing machine learning-based diagnosis models receives great attention from researchers and scientists using histopathology images. Deep learning (DL) algorithms automatically extract features from raw data through convolutional operations. The generalization of the DL models' results relies on large datasets, although they eliminate the expert knowledge in the feature extraction stage. This work aimed to compare the performance of the features extracted via deep learning and a knowledge-based approach in breast cancer detection from histopathology images. Cancer develops when a single or a group of cells grows and spreads uncontrollably. Histopathology images are used in cancer diagnosis since they show tissue and cell structures under a microscope. Knowledge-based and deep learning-based computer-aided detection is an ongoing research field in cancer diagnosis using histopathology images. Feature extraction is vital in both approaches since the feature set is fed to a classifier and determines the performance. This paper evaluates three feature extraction methods and their performance in breast cancer diagnosis. Features are extracted by (1) a Convolutional Neural Network, (2) a transfer learning architecture VGG16, and (3) a knowledge-based system. The feature sets are tested by seven classifiers, including Neural Network (64 units), Random Forest, Multilayer Perceptron, Decision Tree, Support Vector Machines, K-Nearest Neighbors, and Narrow Neural Network (10 units) on the BreakHis 400× image dataset. The CNN achieved up to 85% for the Neural Network and Random Forest, the VGG16 method achieved up to 86% for the Neural Network, and the knowledge-based features achieved up to 98% for Neural Network, Random Forest, Multilayer Perceptron classifiers. [ABSTRACT FROM AUTHOR]

This paper presents a novel facial expression recognition network, called Distract your Attention Network (DAN). Our method is based on two key observations in biological visual perception. Firstly, multiple facial expression classes share inherently similar underlying facial appearance, and their differences could be subtle. Secondly, facial expressions simultaneously exhibit themselves through multiple facial regions, and for recognition, a holistic approach by encoding high-order interactions among local features is required. To address these issues, this work proposes DAN with three key components: Feature Clustering Network (FCN), Multi-head Attention Network (MAN), and Attention Fusion Network (AFN). Specifically, FCN extracts robust features by adopting a large-margin learning objective to maximize class separability. In addition, MAN instantiates a number of attention heads to simultaneously attend to multiple facial areas and build attention maps on these regions. Further, AFN distracts these attentions to multiple locations before fusing the feature maps to a comprehensive one. Extensive experiments on three public datasets (including AffectNet, RAF-DB, and SFEW 2.0) verified that the proposed method consistently achieves state-of-the-art facial expression recognition performance. The DAN code is publicly available. [ABSTRACT FROM AUTHOR]

This paper proposes a method for recognizing the origin of Saposhnikovia divaricata using the IResNet model to achieve computer vision-based classification. Firstly, we created a small sample dataset and applied data augmentation techniques to enhance its diversity. After that, we introduced the hierarchical residual connection block in the early stage of the original model to expand the perceptual field of the neural network and enhance the extraction of scale features. Meanwhile, a depth-separable convolution operation was adopted in the later stage of the model to replace the conventional convolution operation and further reduce the time cost of the model. The experimental results demonstrate that the improved network model achieved a 5.03% improvement in accuracy compared to the original model while also significantly reducing the number of parameters required for the model. In our experiments, we compared the accuracy of the proposed model with several classical convolutional neural network models, including ResNet50, Resnest50, Res2net50, RepVggNet_B0, and ConvNext_T. The results showed that our proposed model achieved an accuracy of 93.72%, which outperformed ResNet50 (86.68%), Resnest50 (89.38%), Res2net50 (91.83%), RepVggNet_B0 (88.68%), and ConvNext_T (92.18%). Furthermore, our proposed model achieved the highest accuracy among the compared models, with a transmission frame rate of 158.9 fps and an inference time of only 6.29 ms. The research methodology employed in this study has demonstrated the ability to reduce potential errors caused by manual observation, effectively improving the recognition ability of Saposhnikovia divaricata based on existing data. Furthermore, the findings of this study provide valuable reference and support for future efforts to develop lightweight models in this area. [ABSTRACT FROM AUTHOR]

Remote sensing change detection involves detecting pixels that have changed from a bi-temporal image of the same location. Current mainstream change detection models use encoder-decoder structures as well as Siamese networks. However, there are still some challenges with this: (1) Existing change feature fusion approaches do not take into account the symmetry of change features, which leads to information loss; (2) The encoder is independent of the change detection task, and feature extraction is performed separately for dual-time images, which leads to underutilization of the encoder parameters; (3) There are problems of unbalanced positive and negative samples and bad edge region detection. To solve the above problems, a mutual feature-aware network (MFNet) is proposed in this paper. Three modules are proposed for the purpose: (1) A symmetric change feature fusion module (SCFM), which uses double-branch feature selection without losing feature information and focuses explicitly on focal spatial regions based on cosine similarity to introduce strong a priori information; (2) A mutual feature-aware module (MFAM), which introduces change features in advance at the encoder stage and uses a cross-type attention mechanism for long-range dependence modeling; (3) A loss function for edge regions. After detailed experiments, the F1 scores of MFNet on SYSU-CD and LEVIR-CD were 83.11% and 91.52%, respectively, outperforming several advanced algorithms, demonstrating the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Due to the sharp increase in household waste, its separate collection is essential in order to reduce the huge amount of household waste, since it is difficult to recycle trash without separate collection. However, since it is costly and time-consuming to separate trash manually, it is crucial to develop an automatic system for separate collection using deep learning and computer vision. In this paper, we propose two Anchor-free-based Recyclable Trash Detection Networks (ARTD-Net) which can recognize overlapped multiple wastes of different types efficiently by using edgeless modules: ARTD-Net1 and ARTD-Net2. The former is an anchor-free based one-stage deep learning model which consists of three modules: centralized feature extraction, multiscale feature extraction and prediction. The centralized feature extraction module in backbone architecture focuses on extracting features around the center of the input image to improve detection accuracy. The multiscale feature extraction module provides feature maps of different scales through bottom-up and top-down pathways. The prediction module improves classification accuracy of multiple objects based on edge weights adjustments for each instance. The latter is an anchor-free based multi-stage deep learning model which can efficiently finds each of waste regions by additionally exploiting region proposal network and RoIAlign. It sequentially performs classification and regression to improve accuracy. Therefore, ARTD-Net2 is more accurate than ARTD-Net1, while ARTD-Net1 is faster than ARTD-Net2. We shall show that our proposed ARTD-Net1 and ARTD-Net2 methods achieve competitive performance in mean average precision and F1 score compared to other deep learning models. The existing datasets have several problems that do not deal with the important class of wastes produced commonly in the real world, and they also do not consider the complex arrangement of multiple wastes with different types. Moreover, most of the existing datasets have an insufficient number of images with low resolution. We shall present a new recyclables dataset which is composed of a large number of high-resolution waste images with additional essential classes. We shall show that waste detection performance is improved by providing various images with the complex arrangement of overlapped multiple wastes with different types. [ABSTRACT FROM AUTHOR]

This paper describes a development that offers new opportunities for detecting faulty bearings. Prioritization is based on the technique for order of preference by similarity to the ideal solution (TOPSIS) for the most discriminative features in the faulty bearing dataset. The proposed model is divided into three steps: feature extraction, feature selection, and classification. In feature extraction, variational mode decomposition (VMD) and fast Fourier transform (FFT) are used to extract features from the measured signal of the test motors and use the symmetrical uncertainty (SU) value for calculation, reducing the redundancy of data. In terms of feature selection, the TOPSIS method is used instead of the traditional filtering method, which is applied to analysis and decision making, and important features are selected from seven filtering methods. Finally, in order to validate the classification ability of the proposed model, k-nearest neighbors (KNN), support vector machine (SVM), and artificial neural networks (ANN) are used as independent classifiers. The effectiveness of the proposed model is evaluated by applying two bearing datasets, namely the current dataset of motor vibration signals and the dataset of bearing motors provided by Case Western Reserve University (CWRU). The results show that the comparison of the proposed model with other models shows the feasibility of this study. [ABSTRACT FROM AUTHOR]

White blood cells (WBCs) constitute an essential part of the human immune system. The correct identification of WBC subtypes is critical in the diagnosis of leukemia, a kind of blood cancer defined by the aberrant proliferation of malignant leukocytes in the bone marrow. The traditional approach of classifying WBCs, which involves the visual analysis of blood smear images, is labor-intensive and error-prone. Modern approaches based on deep convolutional neural networks provide significant results for this type of image categorization, but have high processing and implementation costs owing to very large feature sets. This paper presents an improved hybrid approach for efficient WBC subtype classification. First, optimum deep features are extracted from enhanced and segmented WBC images using transfer learning on pre-trained deep neural networks, i.e., DenseNet201 and Darknet53. The serially fused feature vector is then filtered using an entropy-controlled marine predator algorithm (ECMPA). This nature-inspired meta-heuristic optimization algorithm selects the most dominant features while discarding the weak ones. The reduced feature vector is classified with multiple baseline classifiers with various kernel settings. The proposed methodology is validated on a public dataset of 5000 synthetic images that correspond to five different subtypes of WBCs. The system achieves an overall average accuracy of 99.9 % with more than 95 % reduction in the size of the feature vector. The feature selection algorithm also demonstrates better convergence performance as compared to classical meta-heuristic algorithms. The proposed method also demonstrates a comparable performance with several existing works on WBC classification. [ABSTRACT FROM AUTHOR]

In recent years, deep-learning-based single image super-resolution reconstruction has achieved good performance. However, most existing methods pursue a high peak signal-to-noise ratio (PSNR), while ignoring the quality of the structure and texture details, resulting in unsatisfactory performance of the reconstruction results in terms of human subjective perception. To solve this issue, this paper proposes a structure- and texture-preserving image super-resolution reconstruction method. Specifically, two different network branches are used to extract features for image structure and texture details. A dual-coordinate direction perception attention (DCDPA) mechanism is designed to highlight structure and texture features. The attention mechanism fully considers the complementarity and directionality of multi-scale image features and effectively avoids information loss and possible distortion of image structure and texture details during image reconstruction. Additionally, a cross-fusion mechanism is designed to comprehensively utilize structure and texture information for super-resolution image reconstruction, which effectively integrates the structure and texture details extracted by the two branch networks. Extensive experiments verify the effectiveness of the proposed method and its superiority over existing methods. [ABSTRACT FROM AUTHOR]

Plant point cloud segmentation is an important step in 3D plant phenotype research. Because the stems, leaves, flowers, and other organs of plants are often intertwined and small in size, this makes plant point cloud segmentation more challenging than other segmentation tasks. In this paper, we propose MASPC_Transform, a novel plant point cloud segmentation network base on multi-head attention separation and position code. The proposed MASPC_Transform establishes connections for similar point clouds scattered in different areas of the point cloud space through multiple attention heads. In order to avoid the aggregation of multiple attention heads, we propose a multi-head attention separation loss based on spatial similarity, so that the attention positions of different attention heads can be dispersed as much as possible. In order to reduce the impact of point cloud disorder and irregularity on feature extraction, we propose a new point cloud position coding method, and use the position coding network based on this method in the local and global feature extraction modules of MASPC_Transform. We evaluate our MASPC_Transform on the ROSE_X dataset. Compared with the state-of-the-art approaches, the proposed MASPC_Transform achieved better segmentation results. [ABSTRACT FROM AUTHOR]

As a popular technique for swapping faces with someone else's in images or videos through deep neural networks, deepfake causes a serious threat to the security of multimedia content today. However, because counterfeit images are usually compressed when propagating over the Internet, and because the compression factor used is unknown, most of the existing deepfake detection models have poor robustness for the detection of compressed images with unknown compression factors. To solve this problem, we notice that an image has a high similarity with its compressed image based on symmetry, and this similarity is not easily affected by the compression factor, so this similarity feature can be used as an important clue for compressed deepfake detection. A TCNSC (Two-branch Convolutional Networks with Similarity and Classifier) method that combines compression factor independence is proposed in this paper. The TCNSC method learns two feature representations from the deepfake image, i.e., similarity of the image and its compressed counterpart and authenticity of the deepfake image. A joint training strategy is then utilized for feature extraction, in which the similarity characteristics are obtained by similarity learning while obtaining authenticity characteristics, so the proposed TCNSC model is trained for robust feature learning. Experimental results on the FaceForensics++ (FF++) dataset show that the proposed method significantly outperforms all competing methods under three compression settings of high-quality (HQ), medium-quality (MQ), and low-quality (LQ). For the LQ, MQ, and HQ settings, TCNSC achieves 91.8%, 93.4%, and 95.3% in accuracy, and outperforms the state-of-art method (Xception-RAW) by 16.9%, 10.1%, and 4.1%, respectively. [ABSTRACT FROM AUTHOR]

Deep neural networks have been successfully applied to generate predictive patterns from medical and diagnostic data. This paper presents an approach for assessing persons with Alzheimer's disease (AD) mild cognitive impairment (MCI), compared with normal control (NC) persons, using the zoom-in neural network (ZNN) deep-learning algorithm. ZNN stacks a set of zoom-in learning units (ZLUs) in a feedforward hierarchy without backpropagation. The resting-state fMRI (rs-fMRI) dataset for AD assessments was obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI). The Automated Anatomical Labeling (AAL-90) atlas, which provides 90 neuroanatomical functional regions, was used to assess and detect the implicated regions in the course of AD. The features of the ZNN are extracted from the 140-time series rs-fMRI voxel values in a region of the brain. ZNN yields the three classification accuracies of AD versus MCI and NC, NC versus AD and MCI, and MCI versus AD and NC of 97.7%, 84.8%, and 72.7%, respectively, with the seven discriminative regions of interest (ROIs) in the AAL-90. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing images (HRSI) have the characteristics of foreign objects with the same spectrum. As it is difficult to label samples manually, the hyperspectral remote sensing images are understood to be typical "small sample" datasets. Deep neural networks can effectively extract the deep features from the HRSI, but the classification accuracy mainly depends on the training label samples. Therefore, the stacked convolutional autoencoder network and transfer learning strategy are employed in order to design a new stacked convolutional autoencoder network model transfer (SCAE-MT) for the purposes of classifying the HRSI in this paper. In the proposed classification method, the stacked convolutional au-to-encoding network is employed in order to effectively extract the deep features from the HRSI. Then, the transfer learning strategy is applied to design a stacked convolutional autoencoder network model transfer under the small and limited training samples. The SCAE-MT model is used to propose a new HRSI classification method in order to solve the small samples of the HRSI. In this study, in order to prove the effectiveness of the proposed classification method, two HRSI datasets were chosen. In order to verify the effectiveness of the methods, the overall classification accuracy (OA) of the convolutional self-coding network classification method (CAE), the stack convolutional self-coding network classification method (SCAE), and the SCAE-MT method under 5%, 10%, and 15% training sets are calculated. When compared with the CAE and SCAE models in 5%, 10%, and 15% training datasets, the overall accuracy (OA) of the SCAE-MT method was improved by 2.71%, 3.33%, and 3.07% (on average), respectively. The SCAE-MT method is, thus, clearly superior to the other methods and also shows a good classification performance. [ABSTRACT FROM AUTHOR]

Although gait recognition has been greatly improved by efforts from many researchers in recent years, its performance is still unsatisfactory due to the lack of gait information under the real scenariowhere only one or two images may be used for recognition. In this paper, a new gait recognition framework is brought about which can combine the long-short-term attention modules on silhouette images over the whole sequence and the real human physiological information calculated by a monocular image. The contributions of this work include the following: (1) Fusing the global long-term attention (GLTA) and local short-term attention (LSTA) over the whole query sequence to improve the gait recognition accuracy, where both the short-term gait feature (from two or three frames) and long-term feature (from the whole sequence) are extracted; (2) presenting a method to calculate the real personal static and dynamic physiological features through a single monocular image; (3) by efficiently applying the human physiological information, a new physiological feature extraction (PFE) network is proposed to concatenate the physiological information with silhouette for gait recognition. Through the experiments between the CASIA-B and Multi-state Gait datasets, the effectiveness and efficiency of the proposed method are proven. Under three different walking conditions of the CASIA-B dataset, the mean accuracy of rank-1 in our method is up to 89.6%, and in the Multi-state Gait dataset, wearing different clothes, the mean accuracy of rank-1 in our method is 2.4% higher than the other works. [ABSTRACT FROM AUTHOR]

Currently, the online non-destructive testing (NDT) methods to measure the contact states of high-voltage circuit breakers (HVCBs) with SF6 gas as a quenching medium are lacking. This paper aims to put forward a novel method to detect the contact state of an HVCB based on the vibrational signal. First, for a 40.5-kV SF6 HVCB prototype, a mechanical vibration detection system along with a high-current generator to provide the test current is designed. Given this, vibration test experiments are carried out, and the vibration signal data under various currents and corresponding contact states are obtained. Afterward, a feature extraction method based on the frequency is designed. The state of the HVCB contacts is then determined using optimized deep neural networks (DNNs) along with the method of adaptive moment estimation (Adam) on the obtained experimental data. Finally, the hyperparameters for the DNNs are tuned using the Bayesian optimization (BO) technique, and a global HVCB contact state recognition model at various currents is proposed. The obtained results clearly depict that the proposed recognition model can accurately identify five various contact states of HVCBs for the currents between 1000 A and 3500 A, and the recognition accuracy rate is above 96%. The designed experimental and theoretical analysis in our study will provide the references for future monitoring and diagnosis of faults in HVCBs. [ABSTRACT FROM AUTHOR]

Hyperspectral anomaly detection is a popular research direction for hyperspectral images; however, it is problematic because it separates the background and anomaly without prior target information. Currently, deep neural networks are used as an extractor to mine intrinsic features in hyperspectral images, which can be fed into separate anomaly detection methods to improve their performances. However, this hybrid approach is suboptimal because the subsequent detector is unable to drive the data representation in hidden layers, which makes it a challenge to maximize the capabilities of deep neural networks when extracting the underlying features customized for anomaly detection. To address this issue, a novel unsupervised, self-attention-based, one-class neural network (SAOCNN) is proposed in this paper. SAOCNN consists of two components: a novel feature extraction network and a one-class SVM (OC-SVM) anomaly detection method, which are interconnected and jointly trained by the OC-SVM-like loss function. The adoption of co-training updates the feature extraction network together with the anomaly detector, thus improving the whole network's detection performance. Considering that the prominent feature of an anomaly lies in its difference from the background, we designed a deep neural extraction network to learn more comprehensive hyperspectral image features, including spectral, global correlation, and local spatial features. To accomplish this goal, we adopted an adversarial autoencoder to produce the residual image with highlighted anomaly targets and a suppressed background, which is input into an improved non-local module to adaptively select the useful global information in the whole deep feature space. In addition, we incorporated a two-layer convolutional network to obtain local features. SAOCNN maps the original hyperspectral data to a learned feature space with better anomaly separation from the background, making it possible for the hyperplane to separate them. Our experiments on six public hyperspectral datasets demonstrate the state-of-the-art performance and superiority of our proposed SAOCNN when extracting deep potential features, which are more conducive to anomaly detection. [ABSTRACT FROM AUTHOR]

Loop closure detection is an important component of Simultaneous Localization and Mapping (SLAM). In this paper, a novel two-branch loop closure detection algorithm unifying deep Convolutional Neural Network (ConvNet) features and semantic edge features is proposed. In detail, we use one feature extraction module to extract both ConvNet and semantic edge features simultaneously. The deep ConvNet features are subjected to a Context Feature Enhancement (CFE) module in the global feature ranking branch to generate a representative global feature descriptor. Concurrently, to reduce the interference of dynamic features, the extracted semantic edge information of landmarks is encoded through the Vector of Locally Aggregated Descriptors (VLAD) framework in the semantic edge feature ranking branch to form semantic edge descriptors. Finally, semantic, visual, and geometric information is integrated by the similarity score fusion calculation. Extensive experiments on six public datasets show that the proposed approach can achieve competitive recall rates at 100% precision compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This article aims to propose an algorithm for the automatic recognition of selected radar signals. The algorithm can find application in areas such as Electronic Warfare (EW), where automatic recognition of the type of intra-pulse modulation or the type of emitter operation mode can aid the decision-making process. The simulations carried out included the analysis of the classification possibilities of linear frequency modulated pulsed waveform (LFMPW), stepped frequency modulated pulsed waveform (SFMPW), phase coded pulsed waveform (PCPW), rectangular pulsed waveforms (RPW), frequency modulated continuous wave (FMCW), continuous wave (CW), Stepped Frequency Continuous Wave SFCW) and Phase Coded Continuous Waveform (PCCW). The algorithm proposed in this paper is based on the use of continuous wavelet transform (CWT) coefficients and higher-order statistics (HOS) in the feature determination of selected signals. The Principal Component Analysis (PCA) method was used for dimensionality reduction. An artificial neural network was then used as a classifier. Simulation studies took into account the presence of noise interference with signal-to-noise ratio (SNR) in the range from −5 to 10 dB. Finally, the obtained classification efficiency is presented in the form of a confusion matrix. The simulation results show a high recognition test accuracy, above 99% with a signal-to-noise ratio greater than 0 dB. The article also deals with the selection of the type and parameters of the wavelet. The authors also point to the problems encountered during the research and examples of how to solve them. [ABSTRACT FROM AUTHOR]

Short-term load forecasting (STLF) has a significant role in reliable operation and efficient scheduling of power systems. However, it is still a major challenge to accurately predict power load due to social and natural factors, such as temperature, humidity, holidays and weekends, etc. Therefore, it is very important for the efficient feature selection and extraction of input data to improve the accuracy of STLF. In this paper, a novel hybrid model based on empirical mode decomposition (EMD), a one-dimensional convolutional neural network (1D-CNN), a temporal convolutional network (TCN), a self-attention mechanism (SAM), and a long short-term memory network (LSTM) is proposed to fully decompose the input data and mine the in-depth features to improve the accuracy of load forecasting. Firstly, the original load sequence was decomposed into a number of sub-series by the EMD, and the Pearson correlation coefficient method (PCC) was applied for analyzing the correlation between the sub-series with the original load data. Secondly, to achieve the relationships between load series and external factors during an hour scale and the correlations among these data points, a strategy based on the 1D-CNN and TCN is proposed to comprehensively refine the feature extraction. The SAM was introduced to further enhance the key feature information. Finally, the feature matrix was fed into the long short-term memory (LSTM) for STLF. According to experimental results employing the North American New England Control Area (ISO-NE-CA) dataset, the proposed model is more accurate than 1D-CNN, LSTM, TCN, 1D-CNN–LSTM, and TCN–LSTM models. The proposed model outperforms the 1D-CNN, LSTM, TCN, 1D-CNN–LSTM, and TCN–LSTM by 21.88%, 51.62%, 36.44%, 42.75%, 16.67% and 40.48%, respectively, in terms of the mean absolute percentage error. [ABSTRACT FROM AUTHOR]

Speech signals carry various bits of information relevant to the speaker such as age, gender, accent, language, health, and emotions. Emotions are conveyed through modulations of facial and vocal expressions. This paper conducts an empirical comparison of performances between the classical classifiers: Gaussian Mixture Model (GMM), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Artificial neural networks (ANN); and the deep learning classifiers, i.e., Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Gated Recurrent Unit (GRU) in addition to the ivector approach for a text-independent speaker verification task in neutral and emotional talking environments. The deep models undergo hyperparameter tuning using the Grid Search optimization algorithm. The models are trained and tested using a private Arabic Emirati Speech Database, Ryerson Audio–Visual Database of Emotional Speech and Song dataset (RAVDESS) database, and a public Crowd-Sourced Emotional Multimodal Actors (CREMA) database. Experimental results illustrate that deep architectures do not necessarily outperform classical classifiers. In fact, evaluation was carried out through Equal Error Rate (EER) along with Area Under the Curve (AUC) scores. The findings reveal that the GMM model yields the lowest EER values and the best AUC scores across all datasets, amongst classical classifiers. In addition, the ivector model surpasses all the fine-tuned deep models (CNN, LSTM, and GRU) based on both evaluation metrics in the neutral, as well as the emotional speech. In addition, the GMM outperforms the ivector using the Emirati and RAVDESS databases. [ABSTRACT FROM AUTHOR]

Marine activities occupy an important position in human society. The accurate classification of ships is an effective monitoring method. However, traditional image classification has the problem of low classification accuracy, and the corresponding ship dataset also has the problem of long-tail distribution. Aimed at solving these problems, this paper proposes a fine-grained classification method of optical remote sensing ship images based on deep convolution neural network. We use three-level images to extract three-level features for classification. The first-level image is the original image as an auxiliary. The specific position of the ship in the original image is located by the gradient-weighted class activation mapping. The target-level image as the second-level image is obtained by threshold processing the class activation map. The third-level image is the midship position image extracted from the target image. Then we add self-calibrated convolutions to the feature extraction network to enrich the output features. Finally, the class imbalance is solved by reweighting the class-balanced loss function. Experimental results show that we can achieve accuracies of 92.81%, 93.54% and 93.97%, respectively, after applying the proposed method on different datasets. Compared with other classification methods, this method has a higher accuracy in optical aerospace remote sensing ship classification. [ABSTRACT FROM AUTHOR]

Machine learning (ML) has been widely used worldwide to develop crop yield forecasting models. However, it is still challenging to identify the most critical features from a dataset. Although either feature selection (FS) or feature extraction (FX) techniques have been employed, no research compares their performances and, more importantly, the benefits of combining both methods. Therefore, this paper proposes a framework that uses non-feature reduction (All-F) as a baseline to investigate the performance of FS, FX, and a combination of both (FSX). The case study employs the vegetation condition index (VCI)/temperature condition index (TCI) to develop 21 rice yield forecasting models for eight sub-regions in Vietnam based on ML methods, namely linear, support vector machine (SVM), decision tree (Tree), artificial neural network (ANN), and Ensemble. The results reveal that FSX takes full advantage of the FS and FX, leading FSX-based models to perform the best in 18 out of 21 models, while 2 (1) for FS-based (FX-based) models. These FXS-, FS-, and FX-based models improve All-F-based models at an average level of 21% and up to 60% in terms of RMSE. Furthermore, 21 of the best models are developed based on Ensemble (13 models), Tree (6 models), linear (1 model), and ANN (1 model). These findings highlight the significant role of FS, FX, and specially FSX coupled with a wide range of ML algorithms (especially Ensemble) for enhancing the accuracy of predicting crop yield. [ABSTRACT FROM AUTHOR]

Modulation recognition is the indispensable part of signal interception analysis, which has always been the research hotspot in the field of radio communication. With the increasing complexity of the electromagnetic spectrum environment, interference in signal propagation becomes more and more serious. This paper proposes a modulation recognition scheme based on multimodal feature fusion, which attempts to improve the performance of modulation recognition under different channels. Firstly, different time- and frequency-domain features are extracted as the network input in the signal preprocessing stage. The residual shrinkage building unit with channel-wise thresholds (RSBU-CW) was used to construct deep convolutional neural networks to extract spatial features, which interact with time features extracted by LSTM in pairs to increase the diversity of the features. Finally, the PNN model was adapted to make the features extracted from the network cross-fused to enhance the complementarity between features. The simulation results indicated that the proposed scheme has better recognition performance than the existing feature fusion schemes, and it can also achieve good recognition performance in multipath fading channels. The test results of the public dataset, RadioML2018.01A, showed that recognition accuracy exceeds 95% when the signal-to-noise ratio (SNR) reaches 8dB. [ABSTRACT FROM AUTHOR]

One of the most important and challenging research subjects in computer vision is visual object tracking. The information obtained from the first frame consists of limited and insufficient information to represent an object. If prior information about robust representation that can represent an object well is not sufficient, object tracking fails when not robustly responding to changes in features of the target object according to various factors, namely shape, illumination variation, and scene distortion. In this paper, a real-time single object tracking algorithm is proposed based on a Siamese network to solve this problem. For the object feature extraction, we designed a fully convolutional neural network that removes a fully connected layer and configured a convolution block consisting of a bottleneck structure that preserves the information in a previous layer. This network was designed as a Siamese network, while a regional proposal network was combined at the end of the network for object tracking. The ImageNet Large-Scale Visual Recognition Challenge 2017 dataset was used to train the network in the pre-training phase. Then, in the experimental phase, the object tracking benchmark dataset was used to quantitatively evaluate the network. The experimental results revealed that the proposed tracking algorithm produced more competitive results compared to other tracking algorithms. [ABSTRACT FROM AUTHOR]

Human emotions are variant with time, non-stationary, complex in nature, and are invoked as a result of human reactions during our daily lives. Continuously detecting human emotions from one-dimensional EEG signals is an arduous task. This paper proposes an advanced signal processing mechanism for emotion detection from EEG signals using continuous wavelet transform. The space and time components of the raw EEG signals are converted into 2D spectrograms followed by feature extraction. A hybrid spatio-temporal deep neural network is implemented to extract rich features. A differential-based entropy feature selection technique adaptively differentiates features based on entropy, based on low and high information regions. Bag of Deep Features (BoDF) is applied to create clusters of similar features and computes the features vocabularies for reduction of feature dimensionality. Extensive experiments are performed on the SEED dataset, which shows the significance of the proposed method compared to state-of-the-art methods. Specifically, the proposed model achieved 96.7%, 96.2%, 95.8%, and 95.3% accuracy with the SJTU SEED dataset, for SVM, ensemble, tree, and KNN classifiers, respectively. [ABSTRACT FROM AUTHOR]

With increasing access to open spatial data, it is possible to improve the quality of analyses carried out in the preliminary stages of the investment process. The extraction of buildings from raster data is an important process, especially for urban, planning and environmental studies. It allows, after processing, to represent buildings registered on a given image, e.g., in a vector format. With an actual image it is possible to obtain current information on the location of buildings in a defined area. At the same time, in recent years, there has been huge progress in the use of machine learning algorithms for object identification purposes. In particular, the semantic segmentation algorithms of deep convolutional neural networks which are based on the extraction of features from an image by means of masking have proven themselves here. The main problem with the application of semantic segmentation is the limited availability of masks, i.e., labelled data for training the network. Creating datasets based on manual labelling of data is a tedious, time consuming and capital-intensive process. Furthermore, any errors may be reflected in later analysis results. Therefore, this paper aims to show how to automate the process of data labelling of cadastral data from open spatial databases using convolutional neural networks, and to identify and extract buildings from high resolution orthophotomaps based on this data. The conducted research has shown that automatic feature extraction using semantic ML segmentation on the basis of data from open spatial databases is possible and can provide adequate quality of results. [ABSTRACT FROM AUTHOR]

Change detection (CD) is a particularly important task in the field of remote sensing image processing. It is of practical importance for people when making decisions about transitional situations on the Earth's surface. The existing CD methods focus on the design of feature extraction network, ignoring the strategy fusion and attention enhancement of the extracted features, which will lead to the problems of incomplete boundary of changed area and missing detection of small targets in the final output change map. To overcome the above problems, we proposed a hierarchical attention residual nested U-Net (HARNU-Net) for remote sensing image CD. First, the backbone network is composed of a Siamese network and nested U-Net. We remold the convolution block in nested U-Net and proposed ACON-Relu residual convolution block (A-R), which reduces the missed detection rate of the backbone network in small change areas. Second, this paper proposed the adjacent feature fusion module (AFFM). Based on the adjacency fusion strategy, the module effectively integrates the details and semantic information of multi-level features, so as to realize the feature complementarity and spatial mutual enhancement between adjacent features. Finally, the hierarchical attention residual module (HARM) is proposed, which locally filters and enhances the features in a more fine-grained space to output a much better change map. Adequate experiments on three challenging benchmark public datasets, CDD, LEVIR-CD and BCDD, show that our method outperforms several other state-of-the-art methods and performs excellent in F1, IOU and visual image quality. [ABSTRACT FROM AUTHOR]

In this paper, we propose a deep wavelet neural network (DWNN) model to approximate the natural phenomena that are described by some classical PDEs. Concretely, we introduce wavelets to deep architecture to obtain a fine feature description and extraction. That is, we constructs a wavelet expansion layer based on a family of vanishing momentum wavelets. Second, the Gaussian error function is considered as the activation function owing to its fast convergence rate and zero-centered output. Third, we design the cost function by considering the residual of governing equation, the initial/boundary conditions and an adjustable residual term of observations. The last term is added to deal with the shock wave problems and interface problems, which is conducive to rectify the model. Finally, a variety of numerical experiments are carried out to demonstrate the effectiveness of the proposed approach. The numerical results validate that our proposed method is more accurate than the state-of-the-art approach. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) point cloud classification methods based on deep learning have good classification performance; however, they adapt poorly to diverse datasets and their classification accuracy must be improved. Therefore, FSDCNet, a neural network model based on the fusion of static and dynamic convolution, is proposed and applied for multiview 3D point cloud classification in this paper. FSDCNet devises a view selection method with fixed and random viewpoints, which effectively avoids the overfitting caused by the traditional fixed viewpoint. A local feature extraction operator of dynamic and static convolution adaptive weight fusion was designed to improve the model's adaptability to different types of datasets. To address the problems of large parameters and high computational complexity associated with the current methods of dynamic convolution, a lightweight and adaptive dynamic convolution operator was developed. In addition, FSDCNet builds a global attention pooling, integrating the most crucial information on different view features to the greatest extent. Due to these characteristics, FSDCNet is more adaptable, can extract more fine-grained detailed information, and can improve the classification accuracy of point cloud data. The proposed method was applied to the ModelNet40 and Sydney Urban Objects datasets. In these experiments, FSDCNet outperformed its counterparts, achieving state-of-the-art point cloud classification accuracy. For the ModelNet40 dataset, the overall accuracy (OA) and average accuracy (AA) of FSDCNet in a single view reached 93.8% and 91.2%, respectively, which were superior to those values for many other methods using 6 and 12 views. FSDCNet obtained the best results for 6 and 12 views, achieving 94.6%, 93.3%, 95.3%, and 93.6% in OA and AA metrics, respectively. For the Sydney Urban Objects dataset, FSDCNet achieved an OA and F1 score of 81.2% and 80.1% in a single view, respectively, which were higher than most of the compared methods. In 6 and 12 views, FSDCNet reached an OA of 85.3% and 83.6% and an F1 score of 85.5% and 83.7%, respectively. [ABSTRACT FROM AUTHOR]

To ensure the safe operation of highway traffic lines, given the imperfect feature extraction of existing road pit defect detection models and the practicability of detection equipment, this paper proposes a lightweight target detection algorithm with enhanced feature extraction based on the YOLO (You Only Look Once) algorithm. The BIFPN (Bidirectional Feature Pyramid Network) network structure is used for multi-scale feature fusion to enhance the feature extraction ability, and Varifocal Loss is used to optimize the sample imbalance problem, which improves the accuracy of road defect target detection. In the evaluation test of the model in the constructed PCD1 (Pavement Check Dataset) dataset, the mAP@.5 (mean Average Precision when IoU = 0.5) of the BV-YOLOv5S (BiFPN Varifocal Loss-YOLOv5S) model increased by 4.1%, 3%, and 0.9%, respectively, compared with the YOLOv3-tiny, YOLOv5S, and B-YOLOv5S (BiFPN-YOLOv5S; BV-YOLOv5S does not use the Improved Focal Loss function) models. Through the analysis and comparison of experimental results, it is proved that the proposed BV-YOLOv5S network model performs better and is more reliable in the detection of pavement defects and can meet the needs of road safety detection projects with high real-time and flexibility requirements. [ABSTRACT FROM AUTHOR]

Currently, deep-learning-based methods have been widely used in fault diagnosis to improve the diagnosis efficiency and intelligence. However, most schemes require a great deal of labeled data and many iterations for training parameters. They suffer from low accuracy and over fitting under the few-shot scenario. In addition, a large number of parameters in the model consumes high computing resources, which is far from practical. In this paper, a multi-scale and lightweight Siamese network architecture is proposed for the fault diagnosis with few samples. The architecture proposed contains two main modules. The first part implements the feature vector extraction of sample pairs. It is composed of two lightweight convolutional networks with shared weights symmetrically. Multi-scale convolutional kernels and dimensionality reduction are used in these two symmetric networks to improve feature extraction and reduce the total number of model parameters. The second part takes charge of calculating the similarity of two feature vectors to achieve fault classification. The proposed network is validated by multiple datasets with different loads and speeds. The results show that the model has better accuracy, fewer model parameters and a scale compared to the baseline approach through our experiments. Furthermore, the model is also proven to have good generalization capability. [ABSTRACT FROM AUTHOR]

Visual object tracking for unmanned aerial vehicles (UAV) is widely used in many fields such as military reconnaissance, search and rescue work, film shooting, and so on. However, the performance of existing methods is still not very satisfactory due to some complex factors including viewpoint changing, background clutters and occlusion. The Siamese trackers, which offer a convenient way of formulating the visual tracking problem as a template matching process, have achieved success in recent visual tracking datasets. Unfortunately, these template match-based trackers cannot adapt well to frequent appearance change in UAV video datasets. To deal with this problem, this paper proposes a template-driven Siamese network (TDSiam), which consists of feature extraction subnetwork, feature fusion subnetwork and bounding box estimation subnetwork. Especially, a template library branch is proposed for the feature extraction subnetwork to adapt to the changeable appearance of the target. In addition, a feature aligned (FA) module is proposed as the core of feature fusion subnetwork, which can fuse information in the form of center alignment. More importantly, a method for occlusion detection is proposed to reduce the noise caused by occlusion. Experiments were conducted on two challenging benchmarks UAV123 and UAV20L, the results verified the more competitive performance of our proposed method compared to the existing algorithms. [ABSTRACT FROM AUTHOR]

This paper describes a low-cost, robust, and accurate remote eye-tracking system that uses an industrial prototype smartphone with integrated infrared illumination and camera. Numerous studies have demonstrated the beneficial use of eye-tracking in domains such as neurological and neuropsychiatric testing, advertising evaluation, pilot training, and automotive safety. Remote eye-tracking on a smartphone could enable the significant growth in the deployment of applications in these domains. Our system uses a 3D gaze-estimation model that enables accurate point-of-gaze (PoG) estimation with free head and device motion. To accurately determine the input eye features (pupil center and corneal reflections), the system uses Convolutional Neural Networks (CNNs) together with a novel center-of-mass output layer. The use of CNNs improves the system’s robustness to the significant variability in the appearance of eye-images found in handheld eye trackers. The system was tested with 8 subjects with the device free to move in their hands and produced a gaze bias of 0.72°. Our hybrid approach that uses artificial illumination, a 3D gaze-estimation model, and a CNN feature extractor achieved an accuracy that is significantly (400%) better than current eye-tracking systems on smartphones that use natural illumination and machine-learning techniques to estimate the PoG. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have been widely used to obtain high-level representation in various computer vision tasks. However, in the field of remote sensing, there are not sufficient images to train a useful deep CNN. Instead, we tend to transfer successful pre-trained deep CNNs to remote sensing tasks. In the transferring process, generalization power of features in pre-trained deep CNNs plays the key role. In this paper, we propose two promising architectures to extract general features from pre-trained deep CNNs for remote scene classification. These two architectures suggest two directions for improvement. First, before the pre-trained deep CNNs, we design a linear PCA network (LPCANet) to synthesize spatial information of remote sensing images in each spectral channel. This design shortens the spatial “distance” of target and source datasets for pre-trained deep CNNs. Second, we introduce quaternion algebra to LPCANet, which further shortens the spectral “distance” between remote sensing images and images used to pre-train deep CNNs. With five well-known pre-trained deep CNNs, experimental results on three independent remote sensing datasets demonstrate that our proposed framework obtains state-of-the-art results without fine-tuning and feature fusing. This paper also provides baseline for transferring fresh pre-trained deep CNNs to other remote sensing tasks. [ABSTRACT FROM AUTHOR]

Feature recognition and fault diagnosis plays an important role in equipment safety and stable operation of rotating machinery. In order to cope with the complexity problem of the vibration signal of rotating machinery, a feature fusion model based on information entropy and probabilistic neural network is proposed in this paper. The new method first uses information entropy theory to extract three kinds of characteristics entropy in vibration signals, namely, singular spectrum entropy, power spectrum entropy, and approximate entropy. Then the feature fusion model is constructed to classify and diagnose the fault signals. The proposed approach can combine comprehensive information from different aspects and is more sensitive to the fault features. The experimental results on simulated fault signals verified better performances of our proposed approach. In real two-span rotor data, the fault detection accuracy of the new method is more than 10% higher compared with the methods using three kinds of information entropy separately. The new approach is proved to be an effective fault recognition method for rotating machinery. [ABSTRACT FROM AUTHOR]

Adopting deep learning methods for human activity recognition has been effective in extracting discriminative features from raw input sequences acquired from body-worn sensors. Although human movements are encoded in a sequence of successive samples in time, typical machine learning methods perform recognition tasks without exploiting the temporal correlations between input data samples. Convolutional neural networks (CNNs) address this issue by using convolutions across a one-dimensional temporal sequence to capture dependencies among input data. However, the size of convolutional kernels restricts the captured range of dependencies between data samples. As a result, typical models are unadaptable to a wide range of activity-recognition configurations and require fixed-length input windows. In this paper, we propose the use of deep recurrent neural networks (DRNNs) for building recognition models that are capable of capturing long-range dependencies in variable-length input sequences. We present unidirectional, bidirectional, and cascaded architectures based on long short-term memory (LSTM) DRNNs and evaluate their effectiveness on miscellaneous benchmark datasets. Experimental results show that our proposed models outperform methods employing conventional machine learning, such as support vector machine (SVM) and k-nearest neighbors (KNN). Additionally, the proposed models yield better performance than other deep learning techniques, such as deep believe networks (DBNs) and CNNs. [ABSTRACT FROM AUTHOR]

During recent years, convolutional neural network (CNN)-based methods have been widely applied to hyperspectral image (HSI) classification by mostly mining the spectral variabilities. However, the spatial consistency in HSI is rarely discussed except as an extra convolutional channel. Very recently, the development of pixel pair features (PPF) for HSI classification offers a new way of incorporating spatial information. In this paper, we first propose an improved PPF-style feature, the spatial pixel pair feature (SPPF), that better exploits both the spatial/contextual information and spectral information. On top of the new SPPF, we further propose a flexible multi-stream CNN-based classification framework that is compatible with multiple in-stream sub-network designs. The proposed SPPF is different from the original PPF in its paring pixel selection strategy: only pixels immediately adjacent to the central one are eligible, therefore imposing stronger spatial regularization. Additionally, with off-the-shelf classification sub-network designs, the proposed multi-stream, late-fusion CNN-based framework outperforms competing ones without requiring extensive network configuration tuning. Experimental results on three publicly available datasets demonstrate the performance of the proposed SPPF-based HSI classification framework. [ABSTRACT FROM AUTHOR]

Using motion information of the upper limb to control the prosthetic hand has become a hotspot of current research. The operation of the prosthetic hand must also be coordinated with the user's intention. Therefore, identifying action intention of the upper limb based on motion information of the upper limb is key to controlling the prosthetic hand. Since a wearable inertial sensor bears the advantages of small size, low cost, and little external environment interference, we employ an inertial sensor to collect angle and angular velocity data during movement of the upper limb. Aiming at the action classification for putting on socks, putting on shoes and tying shoelaces, this paper proposes a recognition model based on the Dynamic Time Warping (DTW) algorithm of the motion unit. Based on whether the upper limb is moving, the complete motion data are divided into several motion units. Considering the delay associated with controlling the prosthetic hand, this paper only performs feature extraction on the first motion unit and the second motion unit, and recognizes action on different classifiers. The experimental results reveal that the DTW algorithm based on motion unit bears a higher recognition rate and lower running time. The recognition rate reaches as high as 99.46%, and the average running time measures 8.027 ms. In order to enable the prosthetic hand to understand the grasping intention of the upper limb, this paper proposes a Generalized Regression Neural Network (GRNN) model based on 10-fold cross-validation. The motion state of the upper limb is subdivided, and the static state is used as the sign of controlling the prosthetic hand. This paper applies a 10-fold cross-validation method to train the neural network model to find the optimal smoothing parameter. In addition, the recognition performance of different neural networks is compared. The experimental results show that the GRNN model based on 10-fold cross-validation exhibits a high accuracy rate, capable of reaching 98.28%. Finally, the two algorithms proposed in this paper are implemented in an experiment of using the prosthetic hand to reproduce an action, and the feasibility and practicability of the algorithm are verified by experiment. [ABSTRACT FROM AUTHOR]

The development of automatic emotion detection systems has recently gained significant attention due to the growing possibility of their implementation in several applications, including affective computing and various fields within biomedical engineering. Use of the electroencephalograph (EEG) signal is preferred over facial expression, as people cannot control the EEG signal generated by their brain; the EEG ensures a stronger reliability in the psychological signal. However, because of its uniqueness between individuals and its vulnerability to noise, use of EEG signals can be rather complicated. In this paper, we propose a methodology to conduct EEG-based emotion recognition by using a filtered bispectrum as the feature extraction subsystem and an artificial neural network (ANN) as the classifier. The bispectrum is theoretically superior to the power spectrum because it can identify phase coupling between the nonlinear process components of the EEG signal. In the feature extraction process, to extract the information contained in the bispectrum matrices, a 3D pyramid filter is used for sampling and quantifying the bispectrum value. Experiment results show that the mean percentage of the bispectrum value from 5 × 5 non-overlapped 3D pyramid filters produces the highest recognition rate. We found that reducing the number of EEG channels down to only eight in the frontal area of the brain does not significantly affect the recognition rate, and the number of data samples used in the training process is then increased to improve the recognition rate of the system. We have also utilized a probabilistic neural network (PNN) as another classifier and compared its recognition rate with that of the back-propagation neural network (BPNN), and the results show that the PNN produces a comparable recognition rate and lower computational costs. Our research shows that the extracted bispectrum values of an EEG signal using 3D filtering as a feature extraction method is suitable for use in an EEG-based emotion recognition system. [ABSTRACT FROM AUTHOR]

Location prediction has attracted much attention due to its important role in many location-based services, such as food delivery, taxi-service, real-time bus system, and advertisement posting. Traditional prediction methods often cluster track points into regions and mine movement patterns within the regions. Such methods lose information of points along the road and cannot meet the demand of specific services. Moreover, traditional methods utilizing classic models may not perform well with long location sequences. In this paper, a spatial-temporal-semantic neural network algorithm (STS-LSTM) has been proposed, which includes two steps. First, the spatial-temporal-semantic feature extraction algorithm (STS) is used to convert the trajectory to location sequences with fixed and discrete points in the road networks. The method can take advantage of points along the road and can transform trajectory into model-friendly sequences. Then, a long short-term memory (LSTM)-based model is constructed to make further predictions, which can better deal with long location sequences. Experimental results on two real-world datasets show that STS-LSTM has stable and higher prediction accuracy over traditional feature extraction and model building methods, and the application scenarios of the algorithm are illustrated. [ABSTRACT FROM AUTHOR]

This paper presents a transient power quality (PQ) disturbance classification approach based on a generalized S-transform and probabilistic neural network (PNN). Specifically, the width factor used in the generalized S-transform is feature oriented. Depending on the specific feature to be extracted from the S-transform amplitude matrix, a favorable value is determined for the width factor, with which the S-transform is performed and the corresponding feature is extracted. Four features obtained this way are used as the inputs of a PNN trained for performing the classification of 8 disturbance signals and one normal sinusoidal signal. The key work of this research includes studying the infiuence of the width factor on the S-transform results, investigating the impacts of the width factor on the distribution behavior of features selected for disturbance classification, determining the favorable value for the width factor by evaluating the classification accuracy of PNN. Simulation results tell that the proposed approach significantly enhances the separation of the disturbance signals, improves the accuracy and generalization ability of the PNN, and exhibits the robustness of the PNN against noises. The proposed algorithm also shows good performance in comparison with other reported studies. [ABSTRACT FROM AUTHOR]