Search

Blade icing seriously affects wind turbines' aerodynamic performance and output power. Timely and accurately predicting blade icing status is crucial to improving the economy and safety of wind farms. However, existing blade icing prediction methods cannot effectively solve the problems of unbalanced icing/non-icing data and low prediction accuracy. In order to solve the above problems, this paper proposes a wind turbine blade icing prediction method based on the focal loss function and CNN-Attention-GRU. First, the recursive feature elimination method combined with the physical mechanism of icing is used to extract features highly correlated with blade icing, and a new feature subset is formed through a sliding window algorithm. Then, the focal loss function is utilized to assign more weight to the ice samples with a lower proportion, addressing the significant class imbalance between the ice and non-ice categories. Finally, based on the CNN-Attention-GRU algorithm, a blade icing prediction model is established using continuous 24-h historical data as the input and the icing status of the next 24 h as the output. The model is compared with advanced neural network models. The results show that the proposed method improves the prediction accuracy and F1 score by an average of 6.41% and 4.27%, respectively, demonstrating the accuracy and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper proposes to administer a multi-step artificial intelligence approach with an ensemble of adaptive neural networks (NNs) trained on 50000 samples to identify partial discharge (PD) diagnostic measurements for in-service medium voltage (MV) power cables. To evaluate the performance of the algorithm, a case study was performed on cables deliberately selected to contain both uncomplicated measurements and disruptive irregularities representative of conditions during field testing. Experimental test results prove that the proposed cable-specific adaptation improves PD identification accuracy, with further increment through the NN ensembles. The main contribution of the approach is in both the cable-specific adaption and the NN ensemble being applied to MV cable field measurements. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral–spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the boom of deep neural networks, blind image quality assessment (BIQA) has achieved great processes. However, the current BIQA metrics are limited when evaluating low-quality images as compared to medium-quality and high-quality images, which restricts their applications in real world problems. In this paper, we first identify that two challenges caused by distribution shift and long-tailed distribution lead to the compromised performance on low-quality images. Then, we propose an intermediary enhancement-based bilateral network with iterative training strategy for solving these two challenges. Drawing on the experience of transitive transfer learning, the proposed metric adaptively introduces enhanced intermediary images to transfer more information to low-quality images for mitigating the distribution shift. Our metric also adopts an iterative training strategy to deal with the long-tailed distribution. This strategy decouples feature extraction and score regression for better representation learning and regressor training. It not only transfers the knowledge learned from the earlier stage to the latter stage, but also makes the model pay more attention to long-tailed low-quality images. We conduct extensive experiments on five authentically distorted image quality datasets. The results show that our metric significantly improves the evaluating performance on low-quality images and delivers state-of-the-art intra-dataset results. During generalization tests, our metric also achieves the best cross-dataset performance. [ABSTRACT FROM AUTHOR]

A general deep learning (DL) mechanism for a multiple hidden layer feed-forward neural network contains two parts, i.e., 1) an unsupervised greedy layer-wise training and 2) a supervised fine-tuning which is usually an iterative process. Although this mechanism has been demonstrated in many fields to be able to significantly improve the generalization of neural network, there is no clear evidence to show which one of the two parts plays the essential role for the generalization improvement, resulting in an argument within the DL community. Focusing on this argument, this paper proposes a new DL approach to train multilayer feed-forward neural networks. This approach uses restricted Boltzmann machine (RBM) as the layer-wise training and uses the generalized inverse of a matrix as the supervised fine-tuning. Different from the general deep training mechanism like back-propagation (BP), the proposed approach does not need to iteratively tune the weights, and therefore, has many advantages such as quick training, better generalization, and high understandability, etc. Experimentally, the proposed approach demonstrates an excellent performance in comparison with BP-based DL and the traditional training method for multilayer random weight neural networks. To a great extent, this paper demonstrates that the supervised part plays a more important role than the unsupervised part in DL, which provides some new viewpoints for exploring the essence of DL. [ABSTRACT FROM AUTHOR]

Recent studies have shown that a Deep Convolutional Neural Network (DCNN) trained on a large image dataset can be used as a universal image descriptor and that doing so leads to impressive performance for a variety of image recognition tasks. Most of these studies adopt activations from a single DCNN layer, usually a fully-connected layer, as the image representation. In this paper, we proposed a novel way to extract image representations from two consecutive convolutional layers: one layer is used for local feature extraction and the other serves as guidance to pool the extracted features. By taking different viewpoints of convolutional layers, we further develop two schemes to realize this idea. The first directly uses convolutional layers from a DCNN. The second applies the pre-trained CNN on densely sampled image regions and treats the fully-connected activations of each image region as a convolutional layer's feature activations. We then train another convolutional layer on top of that as the pooling-guidance convolutional layer. By applying our method to three popular visual classification tasks, we find that our first scheme tends to perform better on applications which demand strong discrimination on lower-level visual patterns while the latter excels in cases that require discrimination on category-level patterns. Overall, the proposed method achieves superior performance over existing approaches for extracting image representations from a DCNN. In addition, we apply cross-layer pooling to the problem of image retrieval and propose schemes to reduce the computational cost. Experimental results suggest that the proposed method achieves promising results for the image retrieval task. [ABSTRACT FROM AUTHOR]

This research paper aims to ensure the reliable and efficient operation of grid-connected solid-state transformers (SSTs) by detecting and evaluating various undesirable operating conditions. The study considers different types of faults, including internal faults like open switches and open capacitors, external faults such as symmetrical and asymmetrical faults occurring at various locations of the SST, and abnormalities on the grid side known as sympathetic inrush conditions. To analyze these operating conditions, the secondary current of the high-frequency transformer is normalized and decomposed using the discrete wavelet transform (DWT) and wavelet packet transform (WPT). From the DWT and WPT decomposition at multiple levels, several statistical parameters are calculated. These statistical parameters are carefully selected from different decomposition levels to enhance the effectiveness of the detection algorithm utilizing DWT and WPT. In order to quickly identify and classify all operating conditions that impact the performance of the grid-connected SST, a three-layer feedforward artificial neural network (ANN) is employed, using the selected statistical features. The accuracy and efficiency of the ANN-based classification approach are evaluated by assessing the effectiveness of the statistical features obtained from DWT and WPT. Simulation results have been altered by introducing various noise levels to systematically assess the performance of the proposed algorithms. The average accuracy of the DWT-ANN algorithm stands at 97.89%, while the WPT-ANN algorithm achieves a slightly elevated accuracy level of 98.01%. This notable similarity in accuracy curtails from the careful selection of the wavelet function, decomposition level, and feature sets. [ABSTRACT FROM AUTHOR]

The key role of spectrum sensing in cognitive radios attracted substantial research attention to improve the performance of detectors. We consider a general model for the receiver noise with the potential of generalization towards modeling noise in various environments. This general noise model describes accurately noise characteristics ranging from the Gaussian noise to the severe impulsive noise. However, many previous studies are based on the ideal Gaussian noise, and they cannot capture the non-Gaussian models. To provide a robust detector against different behaviors of the noise in various environments, we employ a convolutional neural network (CNN) compatible with different noise models. The proposed CNN detector is data-driven, and due to its single-dimensional input layer, it is consistent with the received signal and requires no pre-processing. The likelihood ratio test (LRT), the Wald, and the Rao tests for this problem are derived to enrich the paper with comparative evaluations of the proposed CNN and conventional model-based approaches and other neural networks. Although various simulated scenarios substantiate the general superiority and robustness of the CNN-based against impulsive noise and mismatch of parameters, it requires higher computational complexity than other discussed detectors. [ABSTRACT FROM AUTHOR]

The popular directions for automatic modulation classification algorithms investigate how to develop feature extraction methods for further signal classification. In this paper, we propose a mapping algorithm for a manually designed feature extraction method by using the reconstruction component of principal component analysis (PCA), which further extracts discrimination between signal features via a PCA reconstruction component. Two supervised neural network models are studied to achieve the limits of the learning matrix in modulation signal classification. Some experimental results show that different modulation schemes can be obviously classified using matrix mapping for feature extraction. Moreover, the modulation classification accuracy based on the mapping extraction feature, which has a lower SNR requirement for training, is slightly improved compared with some triradial deep learning methods. [ABSTRACT FROM AUTHOR]

In this paper, we aim to build a comprehensive face detection system which provides a one-stop solution to various practical challenges for face detection in realistic scenarios, e.g., detecting faces from multiple-views, faces with occlusions, exaggerated expressions or blurred faces. Moreover, we introduce an automatic data harvest algorithm to effectively improve the generalization performance of the system even when collecting training faces containing various challenging patterns is difficult. In particular, we introduce three critical components to build the system, i.e., a recently widely used deep convolutional neural network (CNN), a novel blur-aware bi-channel network architecture, and a new self-learning mechanism capable of exploiting video contexts continuously. The aforementioned challenges except for detecting blurred faces can potentially be addressed by the CNN component owing its robustness to local deformation of target faces. The more challenging problem of detecting blurred faces is addressed by the bi-channel architecture component which processes blurred and clear faces adaptively. In addition, to address the difficulties in improving the generalization performance of the learning-based face detection system, we introduce a video-context-based self-learning mechanism into the system, which enables the system to continuously enhance its performance by harvesting faces with challenging training patterns automatically. To exploit video context, the detector is applied to massive unlabeled videos, and challenging faces are captured based on temporal inference. These recaptured faces, generally corresponding to one or multiple challenges mentioned above, are fed into the detection system to further improve its performance. Extensive experiments with the proposed detection system provide new state-of-the-art performance on FDDB data set, PASCAL face data set, AFW data set, and WIDER Face data set. [ABSTRACT FROM AUTHOR]

The main challenges of visual object tracking arise from the arbitrary appearance of the objects that need to be tracked. Most existing algorithms try to solve this problem by training a new model to regenerate or classify each tracked object. As a result, the model needs to be initialized and retrained for each new object. In this paper, we propose to track different objects in an object-independent approach with a novel two-flow convolutional neural network (YCNN). The YCNN takes two inputs (one is an object image patch, the other is a larger searching image patch), then outputs a response map which predicts how likely and where the object would appear in the search patch. Unlike the object-specific approaches, the YCNN is actually trained to measure the similarity between the two image patches. Thus, this model will not be limited to any specific object. Furthermore, the network is end-to-end trained to extract both shallow and deep dedicated convolutional features for visual tracking. And once properly trained, the YCNN can be used to track all kinds of objects without further training and updating. As a result, our algorithm is able to run at a very high speed of 45 frames-per-second. The effectiveness of the proposed algorithm can also be proved by the experiments on two popular data sets: OTB-100 and VOT-2014. [ABSTRACT FROM AUTHOR]

This paper proposes a methodology for the diagnosis of electrical system conditions using fractional-order integral transforms for feature extraction. This work proposes three feature extraction algorithms using the Fractional Fourier Transform (FRFT), the Fourier Transform combined with the Mittag-Leffler function, and the Wavelet Transform (WT). Each algorithm extracts data from an electrical system to obtain a set of features that are classified by an Artificial Neural Network to determine the system's condition. The algorithms are utilized in diagnosing two types of electrical machine faults, one in a photovoltaic system, and another in classifying the power quality disturbances (PQDs). An optimization algorithm is suggested to find the optimal orders of integral transforms. The obtained results demonstrate the system's effective diagnosis, displaying superior performance in classifying the faults and PQDs with complex signals. [ABSTRACT FROM AUTHOR]

Purpose: Early detection of bearing faults is an essential task of machine health monitoring. The bearings are one of the vital components of rotary machines. Various methods and techniques were used over the time to diagnose and improve the bearing fault analysis. This review paper includes a fourfold study on the fault analysis. First, the decomposition methods used for signal processing and extracting the required features are discussed under vibrational analysis for health monitoring. Second, the analysis on oil–air lubrication and thermal models used to study the temperature variations are studied. Third, analysis on sensor placement to select an optimal position in collecting the original signals with least possible disturbances was discussed in detail. Finally, in the fourth section, neural networks used for the fault diagnosis is reviewed and analysed. Methods: The convolutional neural network (CNN), Bayesian neural network (BNN), and probabilistic neural network (PNN) are the techniques most commonly used by researchers recently. Among the networks, CNN with modifications is found to provide high accuracy and robustness in identifying and classifying bearing faults even in noisy environments. Results: Discrete wavelet transform (DWT) method for the feature extraction. Daubechies four (db4) with five-layer decomposition method was used to identify the severity of bearing defects. Also, Orthogonal Fuzzy Neighbourhood Discriminant (OFND) features were extracted. This survey analysed existing works on fault diagnosis on spherical roller bearing in different perspectives. Conclusion: Among all the decomposition methods, variational mode decomposition was the most commonly used method and also found to be very effective compared to other methods. The intrinsic time scale method also finds a reasonable part in many research works. Also, various types of neural networks were used in which CNN and modifications to CNN were mostly used to automatically acquire the sensitive fault information from the decomposed signal data. The time-domain waveforms are used widely for feature extraction. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

To facilitate object localization, the existing convolutional neural network (CNN)-based object detection often requires an object proposal method, which, however, may produce inaccurate region proposals and thus impact the performance. To overcome this setback, this paper presents a novel iterative localization refinement method which, undertaken at a mid-layer of a CNN architecture, progressively refines a subset of region proposals in order to match as much ground-truth as possible. In each iteration, the refinement task is cast into a probabilistic framework based on an ingeniously devised probability function. To expedite the computation of the probability function, a divide-and-conquer paradigm is developed by the theorem of total probability. Moreover, an approximate variant based on a refined sampling strategy is also addressed to further reduce the complexity. The proposed ILR method is not only data-driven and free of learning, but it can also be incorporated with many existing CNN-based object detection algorithms, such as Faster R-CNN to enhance the detection accuracy without changing their configurations. Simulations show that the proposed method can improve the main state-of-the-art works on the PASCAL VOC 2007, 2012 and Youtube-Objects data sets. [ABSTRACT FROM AUTHOR]

Crowd counting is an important task in computer vision, which has many applications in video surveillance. Although the regression-based framework has achieved great improvements for crowd counting, how to improve the discriminative power of image representation is still an open problem. Conventional holistic features used in crowd counting often fail to capture semantic attributes and spatial cues of the image. In this paper, we propose integrating semantic information into learning locality-aware feature (LAF) sets for accurate crowd counting. First, with the help of a convolutional neural network, the original pixel space is mapped onto a dense attribute feature map, where each dimension of the pixelwise feature indicates the probabilistic strength of a certain semantic class. Then, LAF built on the idea of spatial pyramids on neighboring patches is proposed to explore more spatial context and local information. Finally, the traditional vector of locally aggregated descriptor (VLAD) encoding method is extended to a more generalized form weighted-VLAD (W-VLAD) in which diverse coefficient weights are taken into consideration. Experimental results validate the effectiveness of our presented method. [ABSTRACT FROM AUTHOR]

With the development of the ubiquitous pedestrian position in environments where global navigation satellite system is unavailable or there is no preinstalled wireless device for positioning, the use of self-contained sensors becomes more important. Thus, this paper proposed a 3-D pedestrian position system combined with commercial microelectromechanical systems microinertial measurement unit (MIMU) and surface electromyography (SEMG) sensor. The signal of the SEMG is used mainly in two aspects: 1) to improve the recognition rate of motion pattern and 2) to obtain the more accurate 3-D velocity contrasted to MIMU. Therefore, satisfied 3-D position accuracy is obtained in two different terrain environments. [ABSTRACT FROM PUBLISHER]

In this paper, we study the challenging problem of categorizing videos according to high-level semantics such as the existence of a particular human action or a complex event. Although extensive efforts have been devoted in recent years, most existing works combined multiple video features using simple fusion strategies and neglected the utilization of inter-class semantic relationships. This paper proposes a novel unified framework that jointly exploits the feature relationships and the class relationships for improved categorization performance. Specifically, these two types of relationships are estimated and utilized by imposing regularizations in the learning process of a deep neural network (DNN). Through arming the DNN with better capability of harnessing both the feature and the class relationships, the proposed regularized DNN (rDNN) is more suitable for modeling video semantics. We show that rDNN produces better performance over several state-of-the-art approaches. Competitive results are reported on the well-known Hollywood2 and Columbia Consumer Video benchmarks. In addition, to stimulate future research on large scale video categorization, we collect and release a new benchmark dataset, called FCVID, which contains 91,223 Internet videos and 239 manually annotated categories. [ABSTRACT FROM PUBLISHER]

Numerous state-of-the-art recommendation frameworks employ deep neural networks in Collaborative Filtering (CF). In this paper, we propose a cross feature fusion neural network (CFFNN) for the enhancement of CF. Existing studies overlook either user preferences for various item features or the relationship between item features and user features. To solve this problem, we construct a cross feature fusion network to enable the fusion of user features and item features as well as a self-attention network to determine users’ preferences for items. Specifically, we design a feature extraction layer with multiple MLP (Multilayer Perceptrons) modules to extract both user features and item features. Then, we introduce a cross feature fusion mechanism for an accurate determination of the relationship between different user-item interactions. The features of users and items are crossly embedded and then fed into a prediction network. The attention mechanism enables the model to focus on more effective features. The effectiveness of CFFNN model is demonstrated through extensive experiments on four real-world datasets. The experimental results indicate that CFFNN significantly outperforms the existing state-of-the-art models, with a relative improvement of 3.0 to 12.1 percent on hit ratio (HR) and normalized discounted cumulative gain (NDCG) compared with the baselines. [ABSTRACT FROM AUTHOR]

Pruning can remove the redundant parameters and structures of Deep Neural Networks (DNNs) to reduce inference time and memory overhead. As one of the important components of DNN, feature maps (FMs) have been widely used in network pruning. However, previous approaches do not fully investigate the discriminative features in FMs, and also do not explicitly utilize all the features associated with each layer in the pruning procedure. In this paper, we explore the discriminative feature of FMs and explicitly investigate the two-adjacent-layer features of each layer to propose a three-phase hierarchical pruning framework, dubbed as FCHP. Firstly, we decompose each FM into several components to extract the discriminative feature. After that, since pruning each layer is related to the FMs of adjacent layers, we explicitly calculate the feature correlation of discriminative features of two adjacent layers, and then use the feature correlation to cluster FMs into several hierarchies to guide subsequent pruning. Finally, we compute the content of discriminative features, and remove channels corresponding to FMs with fewer discriminative features in each hierarchy, respectively. In the experiment, we prune DNNs with the multiple types of architecture on different benchmarks, and the results have achieved the state-of-the-arts in terms of compressed parameters and FLOPs drop. For example, as for ResNet-56 on CIFAR-10, FCHP respectively obtains 50% of parameters and FLOPs reduction with negligible accuracy loss. Besides, as for ResNet-50 on ImageNet, FCHP reduces 40.5% of parameters and 44.1% of FLOPs with 0.43% of Top-1 accuracy drop. [ABSTRACT FROM AUTHOR]

In this paper, we propose a deep neural network-based no-reference (NR) video quality assessment (VQA) method with spatiotemporal feature fusion and hierarchical information integration to evaluate the perceptual quality of videos. First, a feature extraction model is proposed by using 2D and 3D convolutional layers to gradually extract spatiotemporal features from raw video clips. Second, we design a hierarchical branching network to fuse multiframe features, and the feature vectors at each hierarchical level are comprehensively considered during the process of network optimization. Finally, these two modules and quality regression are synthesized into an end-to-end architecture. Experimental results obtained on benchmark VQA databases demonstrate the superiority of our method over other state-of-the-art algorithms. The source code is available online. [ABSTRACT FROM AUTHOR]

The deep neural network has achieved great success in 3D volumetric correspondence. These methods infer the dense displacement or velocity fields directly from the extracted volumetric features without addressing the intrinsic structure correspondence, being prone to shape and pose variations. On the other hand, the spectral maps address the intrinsic structure matching in the low dimensional embedding space, remain less involved in volumetric image correspondence. This paper presents an unsupervised deep volumetric descriptor learning neural network via the low dimensional spectral maps to address the dense volumetric correspondence. The neural network is optimized by a novel criterion on descriptor alignments in the spectral domain regarding the supervoxel graph. Aside from the deep convolved multi-scale features, we explicitly address the supervoxel-wise spatial and cross-channel dependencies to enrich deep descriptors. The dense volumetric correspondence is formulated as the low-dimensional spectral mapping. The proposed approach has been applied to both synthetic and clinically obtained cone-beam computed tomography images to establish dense supervoxel-wise and up-scaled voxel-wise correspondences. Extensive series of experimental results demonstrate the contribution of the proposed approach in volumetric descriptor extraction and consistent correspondence, facilitating attribute transfer for segmentation and landmark location. The proposed approach performs favorably against the state-of-the-art volumetric descriptors and the deep registration models, being resilient to pose or shape variations and independent of the prior transformations. [ABSTRACT FROM AUTHOR]

Online recognition of gestures is critical for intuitive human–robot interaction (HRI) and further push collaborative robotics into the market, making robots accessible to more people. The problem is that it is difficult to achieve accurate gesture recognition in real unstructured environments, often using distorted and incomplete multisensory data. This paper introduces an HRI framework to classify large vocabularies of interwoven static gestures (SGs) and dynamic gestures (DGs) captured with wearable sensors. DG features are obtained by applying data dimensionality reduction to raw data from sensors (resampling with cubic interpolation and principal component analysis). Experimental tests were conducted using the UC2017 hand gesture dataset with samples from eight different subjects. The classification models show an accuracy of 95.6% for a library of 24 SGs with a random forest and 99.3% for 10 DGs using artificial neural networks. These results compare equally or favorably with different commonly used classifiers. Long short-term memory deep networks achieved similar performance in online frame-by-frame classification using raw incomplete data, performing better in terms of accuracy than static models with specially crafted features, but worse in training and inference time. The recognized gestures are used to teleoperate a robot in a collaborative process that consists in preparing a breakfast meal. [ABSTRACT FROM AUTHOR]

Wafer map defect pattern contains the critical information about semiconductor manufacturing, effective defect analysis technology can improve the yield of products. This paper develops a wafer defect pattern recognition and analysis method based on convolutional neural network (CNN). We build an 8-layer CNN model to inspect wafer map defect and a 13-layer model to classify defect patterns. To analyze the defects, we first extract features based on the classification network, then perform PCA dimensionality reduction and similarity ranking, we infer the root causes of tested samples according to the retrieved wafer maps. In particular, we analyze the impacts of different network layers and feature dimensions on image retrieval performance, it turns out that the appropriate dimensionality reduction can increase the accuracy and speed of wafer map retrieval. The models are trained and tested on the WM-811K wafer map dataset, which are collected from real wafer maps. The experimental results show that the proposed method is able to identify common wafer defect patterns and analyze the root causes accurately and effectively. [ABSTRACT FROM AUTHOR]

In this paper, the well-known bag-of-features (BoFs) model is generalized and formulated as a neural network that is composed of three layers: 1) a radial basis function (RBF) layer; 2) an accumulation layer; and 3) a fully connected layer. This formulation allows for decoupling the representation size from the number of used codewords, as well as for better modeling the feature distribution using a separate trainable scaling parameter for each RBF neuron. The resulting network, called retrieval-oriented neural BoF (RN-BoF), is trained using regular back propagation and allows for fast extraction of compact image representations. It is demonstrated that the RN-BoF model is capable of: 1) increasing the object encoding and retrieval speed; 2) reducing the extracted representation size; and 3) increasing the retrieval precision. A symmetry-aware spatial segmentation technique is also proposed to further reduce the encoding time and the storage requirements and allows the method to efficiently scale to large datasets. The proposed method is evaluated and compared to other state-of-the-art techniques using five different image datasets, including the large-scale YouTube Faces database. [ABSTRACT FROM AUTHOR]

In general, an image attribute is a human-nameable visual property that has a semantic connotation. Appropriate modeling of the intrinsic contextual correlations among attributes plays a fundamental role in attribute detection. In this paper, we consider image attribute detection from the perspective of regularized deep learning. In particular, we propose a regularized deep belief network (rDBN) to perform the image attribute detection task, which is composed of two parts: 1) a detection DBN (dDBN) that models the joint distribution of images and their corresponding attributes, which acts as an attribute detector and 2) a contextual restricted Boltzmann machine that explicitly models the correlations among attributes acting as a regularizer that restraints the output detection result given by the dDBN to meet the contextual prior of attributes. Furthermore, we propose an efficient fine-tuning scheme that can further optimize the performance of the dDBN by backpropagation. Experimental results show that the proposed rDBN obtains improvements over the state-of-the-art methods for attribute detection on the benchmark data sets. [ABSTRACT FROM PUBLISHER]

In this paper, we investigate Hebbian learning strategies applied to Convolutional Neural Network (CNN) training. We consider two unsupervised learning approaches, Hebbian Winner-Takes-All (HWTA), and Hebbian Principal Component Analysis (HPCA). The Hebbian learning rules are used to train the layers of a CNN in order to extract features that are then used for classification, without requiring backpropagation (backprop). Experimental comparisons are made with state-of-the-art unsupervised (but backprop-based) Variational Auto-Encoder (VAE) training. For completeness,we consider two supervised Hebbian learning variants (Supervised Hebbian Classifiers—SHC, and Contrastive Hebbian Learning—CHL), for training the final classification layer, which are compared to Stochastic Gradient Descent training. We also investigate hybrid learning methodologies, where some network layers are trained following the Hebbian approach, and others are trained by backprop. We tested our approaches on MNIST, CIFAR10, and CIFAR100 datasets. Our results suggest that Hebbian learning is generally suitable for training early feature extraction layers, or to retrain higher network layers in fewer training epochs than backprop. Moreover, our experiments show that Hebbian learning outperforms VAE training, with HPCA performing generally better than HWTA. [ABSTRACT FROM AUTHOR]

Recently, deep learning techniques have substantially boosted the performance of salient object detection in still images. However, the salient object detection in videos by using traditional handcrafted features or deep learning features is not fully investigated, probably due to the lack of sufficient manually labeled video data for saliency modeling, especially for the data-driven deep learning. This paper proposes a novel weakly supervised approach to the salient object detection in a video, which can learn a robust saliency prediction model by using very limited manually labeled data and a large amount of weakly labeled data that could be easily generated in a supervised approach. Furthermore, we propose a spatiotemporal cascade neural network architecture for saliency modeling, in which two fully convolutional networks are cascaded to evaluate the visual saliency from both spatial and temporal cues to lead the optimal video saliency prediction. The proposed approach is extensively evaluated on the widely used challenging data sets, and the experiments demonstrate that our proposed approach substantially outperforms the state-of-the-art salient object detection models. [ABSTRACT FROM AUTHOR]

Image representation based on sparse coding generalizes the bag of words model. Although it reduces the reconstruction error for local features to achieve the state-of-the-art image classification performance, the large computational cost hinders the application of sparse coding-based image features. In this paper, we propose approximating a sparse code using the output of a simple neural network. The resulting parameter learning model for the neural network automatically incorporates non-negative and shift-invariant constraints, leading to an efficient normalized non-negative sparse coding (N3SC) sparse encoder. Without the use of the traditional iterative process to solve the sparse coding objective, the sparse encoder directly “converts” each local feature into a sparse code. We also introduce a method for training the encoder based on the auto-encoder method. In addition, we formally propose the corresponding sparse coding scheme called N3SC, which enforces both the non-negative constraint and the shift-invariant constraint in addition to the traditional sparse coding criteria. As demonstrated by several experiments, the obtained N3SC encoder requires only 3%–10% of the processing time for image feature extraction compared with the standard sparse coding scheme. At the same time, the features extracted using the exact solutions of the N3SC coding scheme and the N3SC encoder offer superior image classification accuracy compared to the accuracy of many existing sparse coding-based representations. [ABSTRACT FROM AUTHOR]

This paper proposes a method for vehicle driving behavior recognition based on a six-axis motion processor. This method uses deep-learning technology to learn the sample data collected by the on-board sensor. To solve the problem of small sample size and easy overfitting, we propose a joint data augmentation (JDA) scheme, and design a new multi-view convolutional neural network model (MV-CNN). The JDA includes the multi-axis weighted fusion algorithm, background noise fusion algorithm, and random cropping algorithm to construct a sample dataset that is more in line with a complex real driving environment. With the CNN model, the direction of information propagation improved, and a new MV-CNN model was developed for the training, learning, and recognition of driving behavior. The performance of MV-CNN is experimentally compared with CNN, recurrent neural networks (RNN), LSTM, CNN+LSTM, and three-dimensional CNN. The results show that MV-CNN can obtain the best recall, precision, and F1-score. At the same time, MV-CNN and JDA have better generalization ability, reduce the training variance and deviation, and increase the stability of the model training process. [ABSTRACT FROM AUTHOR]

Defocus blur detection is an important and challenging task in computer vision and digital imaging fields. Previous work on defocus blur detection has put a lot of effort into designing local sharpness metric maps. This paper presents a simple yet effective method to automatically obtain the local metric map for defocus blur detection, which based on the feature learning of multiple convolutional neural networks (ConvNets). The ConvNets automatically learn the most locally relevant features at the super-pixel level of the image in a supervised manner. By extracting convolution kernels from the trained neural network structures and processing it with principal component analysis, we can automatically obtain the local sharpness metric by reshaping the principal component vector. Meanwhile, an effective iterative updating mechanism is proposed to refine the defocus blur detection result from coarse to fine by exploiting the intrinsic peculiarity of the hyperbolic tangent function. The experimental results demonstrate that our proposed method consistently performed better than the previous state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Aiming at improving the classification accuracy with limited numbers of labeled pixels in polarimetric synthetic aperture radar (PolSAR) image classification task, this paper presents a graph-based semisupervised deep learning model for PolSAR image classification. It models the PolSAR image as an undirected graph, where the nodes correspond to the labeled and unlabeled pixels, and the weighted edges represent similarities between the pixels. Upon the graph, we design an energy function incorporating a semisupervision term, a convolutional neural network (CNN) term, and a pairwise smoothness term. The employed CNN extracts abstract and data-driven polarimetric features and outputs class label predictions to the graph model. The semisupervision term enforces the category label constraints on the human-labeled pixels. The pairwise smoothness term encourages class label smoothness and the alignment of class label boundaries with the image edges. Starting from an initialized class label map generated based on $K$ -Wishart distribution hypothesis or superpixel segmentation of PauliRGB images, we iteratively and alternately optimize the defined energy function until it converges. We conducted experiments on two real benchmark PolSAR images, and extensive experiments demonstrated that our approach achieved the state-of-the-art results for PolSAR image classification. [ABSTRACT FROM AUTHOR]

Conversations in the intelligent vehicles are usually short utterance. As the durations of the short utterances are small (e.g., less than 3 s), it is difficult to learn sufficient information to distinguish the type of languages. In this paper, we propose an end-to-end short utterances based speech language identification (SLI) approach, which is especially suitable for the short utterance based language identification. This approach is implemented with a long short-term memory (LSTM) neural network, which is designed for the SLI application in intelligent vehicles. The features used for LSTM learning are generated by a transfer learning method. The bottleneck features of a deep neural network, which are obtained for a mandarin acoustic-phonetic classifier, are used for the LSTM training. In order to improve the SLD accuracy with short utterances, a phase vocoder based time-scale modification method is utilized to reduce/increase the speech rate of the test utterance. By connecting the normal, speech rate reduced, and speech rate increased utterances, we can extend the length of the test utterances such that the performance of the SLI system is improved. The experimental results on the AP17-OLR database demonstrate that the proposed method can improve the performance of SLD, especially on short utterance. The proposed SLI has robust performance under the vehicular noisy environment. [ABSTRACT FROM AUTHOR]

This paper presents an energy-efficient static random access memory (SRAM) with embedded dot-product computation capability, for binary-weight convolutional neural networks. A 10T bit-cell-based SRAM array is used to store the 1-b filter weights. The array implements dot-product as a weighted average of the bitline voltages, which are proportional to the digital input values. Local integrating analog-to-digital converters compute the digital convolution outputs, corresponding to each filter. We have successfully demonstrated functionality (>98% accuracy) with the 10 000 test images in the MNIST hand-written digit recognition data set, using 6-b inputs/outputs. Compared to conventional full-digital implementations using small bitwidths, we achieve similar or better energy efficiency, by reducing data transfer, due to the highly parallel in-memory analog computations. [ABSTRACT FROM AUTHOR]

Action recognition has been extensively researched in computer vision due to its potential applications in a broad range of areas. The key to action recognition lies in modeling actions and measuring their similarity, which however poses great challenges. In this paper, we propose learning match kernels between actions on Grassmann manifold for action recognition. Specifically, we propose modeling actions as a linear subspace on the Grassmann manifold; the subspace is a set of convolutional neural network (CNN) feature vectors pooled temporally over frames in semantic video clips, which simultaneously captures local discriminant patterns and temporal dynamics of motion. To measure the similarity between actions, we propose Grassmann match kernels (GMK) based on canonical correlations of linear subspaces to directly match videos for action recognition; GMK is learned in a supervised way via kernel target alignment, which is endowed with a great discriminative ability to distinguish actions from different classes. The proposed approach leverages the strengths of CNNs for feature extraction and kernels for measuring similarity, which accomplishes a general learning framework of match kernels for action recognition. We have conducted extensive experiments on five challenging realistic data sets including Youtube, UCF50, UCF101, Penn action, and HMDB51. The proposed approach achieves high performance and substantially surpasses the state-of-the-art algorithms by large margins, which demonstrates the great effectiveness of proposed approach for action recognition. [ABSTRACT FROM AUTHOR]

Age-related macular degeneration is one of the main causes of blindness. However, the internal structures of retinas are complex and difficult to be recognized due to the occurrence of neovascularization. Traditional surface detection methods may fail in the layer segmentation. In this paper, a supervised method is reported for simultaneously segmenting layers and neovascularization. Three spatial features, seven gray-level-based features, and 14 layer-like features are extracted for the neural network classifier. The coarse surfaces of different optical coherence tomography (OCT) images can thus be found. To describe and enhance retinal layers with different thicknesses and abnormalities, multi-scale bright and dark layer detection filters are introduced. A constrained graph search algorithm is also proposed to accurately detect retinal surfaces. The weights of nodes in the graph are computed based on these layer-like responses. The proposed method was evaluated on 42 spectral-domain OCT images with age-related macular degeneration. The experimental results show that the proposed method outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

To avoid the exhaustive search over locations and scales, current state-of-the-art object detection systems usually involve a crucial component generating a batch of candidate object proposals from images. In this paper, we present a simple yet effective approach for segmenting object proposals via a deep architecture of recursive neural networks (ReNNs), which hierarchically groups regions for detecting object candidates over scales. Unlike traditional methods that mainly adopt fixed similarity measures for merging regions or finding object proposals, our approach adaptively learns the region merging similarity and the objectness measure during the process of hierarchical region grouping. Specifically, guided by a structured loss, the ReNN model jointly optimizes the cross-region similarity metric with the region merging process as well as the objectness prediction. During inference of the object proposal generation, we introduce randomness into the greedy search to cope with the ambiguity of grouping regions. Extensive experiments on standard benchmarks, e.g., PASCAL VOC and ImageNet, suggest that our approach is capable of producing object proposals with high recall while well preserving the object boundaries and outperforms other existing methods in both accuracy and efficiency. [ABSTRACT FROM AUTHOR]

Person re-identification (Re-ID) aims to match person images captured from two non-overlapping cameras. In this paper, a deep hybrid similarity learning (DHSL) method for person Re-ID based on a convolution neural network (CNN) is proposed. In our approach, a light CNN learning feature pair for the input image pair is simultaneously extracted. Then, both the elementwise absolute difference and multiplication of the CNN learning feature pair are calculated. Finally, a hybrid similarity function is designed to measure the similarity between the feature pair, which is realized by learning a group of weight coefficients to project the elementwise absolute difference and multiplication into a similarity score. Consequently, the proposed DHSL method is able to reasonably assign complexities of feature learning and metric learning in a CNN, so that the performance of person Re-ID is improved. Experiments on three challenging person Re-ID databases, QMUL GRID, VIPeR, and CUHK03, illustrate that the proposed DHSL method is superior to multiple state-of-the-art person Re-ID methods. [ABSTRACT FROM AUTHOR]

This paper proposes a sparse representation and fully connected neural network (SRFCNN) methodology for residential ac series arc fault identification. The SRFCNN method captures the features of idiographic signal using sparse coding and conducts intelligent feature learning and classification through neural network (NN). The methodology is elaborately structured by a pretreatment layer, a sparse representation layer, and a decision layer. The processing procedure includes the following main steps. A set of customized bases are first designed via dictionary learning to behave as precise feature descriptors rather than choosing a fixed dictionary. The pretreated input samples are accordingly transformed into a series of sparse coefficients and then fed into the NN for recognition. Experimental results indicate that the method works well in distinguishing the signature of the arcing state from that of normal state within a certain single load type, as well as recognizing the two states of various load types (resistive, inductive, capacitive, switching, etc.) all together. The general classification accuracy of every half-cycle signal in the 10 tested classes could reach 94.3% and be probably higher. The methodology’s good generality and reliability for arc fault identification show its potential in other unsupervised signal feature recognition and fault detection. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel end-to-end deep neural network model for omnidirectional depth estimation from a wide-baseline multi-view stereo setup. The images captured with ultra-wide field-of-view cameras on an omnidirectional rig are processed by the feature extraction module, and then the deep feature maps are warped onto the concentric spheres swept through all candidate depths using the calibrated camera parameters. The 3D encoder-decoder block takes the aligned feature volume to produce an omnidirectional depth estimate with regularization on uncertain regions utilizing the global context information. For more accurate depth estimation we also propose an uncertainty prior guidance in two ways: depth map filtering and guiding regularization. In addition, we present large-scale synthetic datasets for training and testing omnidirectional multi-view stereo algorithms. Our datasets consist of 13K ground-truth depth maps and 53K fisheye images in four orthogonal directions with various objects and environments. Experimental results show that the proposed method generates excellent results in both synthetic and real-world environments, and it outperforms the prior art and the omnidirectional versions of the state-of-the-art conventional stereo algorithms. [ABSTRACT FROM AUTHOR]

In this paper, a novel perceptual image hashing scheme based on convolutional neural network (CNN) with multiple constraints is proposed, in which our deep hashing network learns the process of features extraction automatically according to the training target and then generates the final hash sequence. The combination of convolutional and pooling layers is to reduce the size of input image while deepening the channels. Then, we construct two pairs of constraints and integrate them into an overall constraint function through a strategy of weight allocation. In order to guarantee the robustness and discrimination of deep hashing network simultaneously, a new training method is developed to adjust the training set structure dynamically according to the changes of constraint values. Experimental results show that the proposed deep hashing network can achieve a satisfactory balance between perceptual robustnzess and discrimination while maintaining security. Based on the large-scale test set, receiver operating characteristic (ROC) curves, ${F}_{1}$ scores and equal error rate (EER) demonstrate the superiority of our scheme in terms of content authentication compared with some state-of-the-art schemes. [ABSTRACT FROM AUTHOR]

This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervision. Instead, we use the same training data as traditional CNNs. Our method automatically assigns each interpretable filter in a high conv-layer with an object part of a certain category during the learning process. Such explicit knowledge representations in conv-layers of the CNN help people clarify the logic encoded in the CNN, i.e., answering what patterns the CNN extracts from an input image and uses for prediction. We have tested our method using different benchmark CNNs with various architectures to demonstrate the broad applicability of our method. Experiments have shown that our interpretable filters are much more semantically meaningful than traditional filters. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Mechanical Engineering / Strojniški Vestnik is the property of University of Ljubljana, Faculty of Mechanical Engineering, Journal of Mechanical Engineering and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Control chart pattern analysis is a crucial task in statistical process control. There are various types of nonrandom patterns that may appear on the control chart indicating that the process is out of control. The presence of nonrandom patterns manifests that a process is affected by assignable causes, and corrective actions should be taken. From a process control point of view, identification of nonrandom patterns can provide clues to the set of possible causes that must be searched; hence, the troubleshooting time could be reduced in length. In this paper, we discuss two implementation modes of control chart pattern recognition and introduce a new research issue concerning pattern displacement problem in the process of control chart analysis. This paper presents a neural network-based pattern recognizer with selected features as inputs. We propose a novel application of statistical correlation analysis for feature extraction purposes. Unlike previous studies, the proposed features are developed by taking the pattern displacement into account. The superior performance of the feature-based recognizer over the raw data-based one is demonstrated using synthetic pattern data. [ABSTRACT FROM AUTHOR]

Jointly combining the spatial and spectral features has proved to dramatically improve the performance of classifying hyperspectral image. Recently, utilizing neural networks to automatically model the spatial-spectral feature representations for hyperspectral images has become of great interest. This paper proposes a simple but innovative framework to classify hyperspectral image with two shallow convolutional neural networks (CNNs). First, principal component analysis whitening is applied to decorrelate hundreds of spectral bands. Instead of selecting the principal components to reduce the spectral dimensionality, we retain all the spectral bands but compress the image cuboid into a one-channel spectral quilt by stacking spectral patches. In this way, not only all the spectral information is retained but also the computational complexity of training a neural network is reduced compared with conventional networks that directly input the spectral volumes. Moreover, the spectral quilt will contain some novel textural patterns that are effective at distinguishing classes. Two shallow CNNs are then applied to classify the spectral quilts. As shown in the experiments, both networks can outperform the standard analysis methods. [ABSTRACT FROM AUTHOR]

This paper presents a system for the detection of ships and oil spills using side-looking airborne radar (SLAR) images. The proposed method employs a two-stage architecture composed of three pairs of convolutional neural networks (CNNs). Each pair of networks is trained to recognize a single class (ship, oil spill, and coast) by following two steps: a first network performs a coarse detection, and then, a second specialized CNN obtains the precise localization of the pixels belonging to each class. After classification, a postprocessing stage is performed by applying a morphological opening filter in order to eliminate small look-alikes, and removing those oil spills and ships that are surrounded by a minimum amount of coast. Data augmentation is performed to increase the number of samples, owing to the difficulty involved in obtaining a sufficient number of correctly labeled SLAR images. The proposed method is evaluated and compared to a single multiclass CNN architecture and to previous state-of-the-art methods using accuracy, precision, recall, F-measure, and intersection over union. The results show that the proposed method is efficient and competitive, and outperforms the approaches previously used for this task. [ABSTRACT FROM AUTHOR]