Search

A gearbox is a critical rotating component that is used to transmit torque from one shaft to another. This paper presents a data-driven gearbox fault diagnosis system in which the issue of variable working conditions namely uneven speed and the load of the machinery is addressed. Moreover, a mechanism is suggested that how an improved feature extraction process and data from multiple tasks can contribute to the overall performance of a fault diagnosis model. The variable working conditions make a gearbox fault diagnosis a challenging task. The performance of the existing algorithms in the literature deteriorates under variable working conditions. In this paper, a refined feature extraction technique and multitask learning are adopted to address this variability issue. The feature extraction step helps to explore unique fault signatures which are helpful to perform gearbox fault diagnosis under uneven speed and load conditions. Later, these extracted features are provided to a convolutional neural network (CNN) based multitask learning (MTL) network to identify the faults in the provided gearbox dataset. A comparison of the experimental results of the proposed model with that of several already published state-of-the-art diagnostic techniques suggests the superiority of the proposed model under uneven speed and load conditions. Therefore, based on the results the proposed approach can be used for gearbox fault diagnosis under uneven speed and load conditions.

Previous research on 3D skeleton-based human action recognition has frequently relied on a sequence-wise viewpoint normalization process, which adjusts the view directions of all segmented action sequences. This type of approach typically demonstrates robustness against variations in viewpoint found in short-term videos, a characteristic commonly encountered in public datasets. However, our preliminary investigation of complex action sequences, such as discussions or smoking, reveals its limitations in capturing the intricacies of such actions. To address these view-dependency issues, we propose a straightforward, yet effective, sequence-wise augmentation technique. This strategy enhances the robustness of action recognition models, particularly against changes in viewing direction that mainly occur within the horizontal plane (azimuth) by rotating human key points around either the z-axis or the spine vector, effectively creating variations in viewing directions. We scrutinize the robustness of this approach against real-world viewpoint variations through extensive empirical studies on multiple public datasets, including an additional set of custom action sequences. Despite the simplicity of our approach, our experimental results consistently yield improved action recognition accuracies. Compared to the sequence-wise viewpoint normalization method used with advanced deep learning models like Conv1D, LSTM, and Transformer, our approach showed a relative increase in accuracy of 34.42% for the z-axis and 10.86% for the spine vector.

Pest counting, which predicts the number of pests in the early stage, is very important because it enables rapid pest control, reduces damage to crops, and improves productivity. In recent years, light traps have been increasingly used to lure and photograph pests for pest counting. However, pest images have a wide range of variability in pest appearance owing to severe occlusion, wide pose variation, and even scale variation. This makes pest counting more challenging. To address these issues, this study proposes a new pest counting model referred to as multiscale and deformable attention CenterNet (Mada-CenterNet) for internal low-resolution (LR) and high-resolution (HR) joint feature learning. Compared with the conventional CenterNet, the proposed Mada-CenterNet adopts a multiscale heatmap generation approach in a two-step fashion to predict LR and HR heatmaps adaptively learned to scale variations, that is, changes in the number of pests. In addition, to overcome the pose and occlusion problems, a new between-hourglass skip connection based on deformable and multiscale attention is designed to ensure internal LR and HR joint feature learning and incorporate geometric deformation, thereby resulting in improved pest counting accuracy. Through experiments, the proposed Mada-CenterNet is verified to generate the HR heatmap more accurately and improve pest counting accuracy owing to multiscale heatmap generation, joint internal feature learning, and deformable and multiscale attention. In addition, the proposed model is confirmed to be effective in overcoming severe occlusions and variations in pose and scale. The experimental results show that the proposed model outperforms state-of-the-art crowd counting and object detection models.

Operational sea ice maps are usually generated manually using dual-polarization (DP) synthetic aperture radar (SAR) satellite imagery, but there is strong interest in automating this process. Recently launched satellites offer compact polarimetry (CP) imagery that provides more comprehensive polarimetric information compared to DP, which compels the use of CP for automated classification of SAR sea ice imagery. Existing sea ice scene classification algorithms using CP imagery rely on handcrafted features, while neural networks offer the potential of features that are more discriminating. We have developed a new and effective sea ice classification algorithm that leverages the nature of CP data. First, a residual-based convolutional neural network (ResCNN) is implemented to classify each pixel. In parallel, an unsupervised segmentation is performed to generate regions based on CP statistical properties. Regions are assigned a single class label by majority voting using the ResCNN output. For testing, quad-polarimetric (QP) SAR sea ice scenes from the RADARSAT Constellation Mission (RCM) are used, and QP, DP, CP, and reconstructed QP modes are compared for classification accuracy, while also comparing them to other classification approaches. Using CP achieves an overall accuracy of 96.86%, which is comparable to QP (97.16%), and higher than reconstructed QP and DP data by about 2% and 10%, respectively. The implemented algorithm using CP imagery provides an improved option for automated sea ice mapping.

Radar-based sensing of gestures has gained tremendous attention with the recent advancements in radar technologies. However, evident discrepancies exist in the gesture samples due to hand flexibility and individual habits. It is challenging for traditional methods to identify the gestures from unknown data sources. Cross-person (Cross-scenario) recognition refers to a recognition where the training and test samples are from different people (scenarios), respectively. To explore how the recognition performance is affected by the individual habits, the reasons are analyzed and visualized through the experiments. On this basis, HandNet is targeted proposed for the low personality-sensitive feature learning and it has two main contributions. First, a Stepped Data Augmentation (SDA) is proposed to reduce the sample interferences by non-coherent accumulating, and capture the inter-frame dependencies. Second, a Focus on Generalization loss (FoG loss) is proposed to highlight the generalized feature learning by res tricting the distances of inter-source features. Extensive experiments demonstrate that HandNet effectively reduces the classifier's sensitivity to the personalized habits, and outperforms the existing state-of-the-art methods on the cross-person and cross-scenario gesture recognition. To the best of our knowledge, it is the first time to dedicate to addressing the radar-based gesture recognition with low personal sensitivity, which is more suitable for practical scenarios. [ABSTRACT FROM AUTHOR]

In the last decades, data-driven methods have gained great popularity in the industry, supported by state-of-the-art advancements in machine learning. These methods require a large quantity of labeled data, which is difficult to obtain and mostly costly and challenging. To address these challenges, researchers have turned their attention to unsupervised and few-shot learning methods, which produced encouraging results, particularly in the areas of computer vision and natural language processing. With the lack of pretrained models, time series feature learning is still considered as an open area of research. This paper presents an efficient two-stage feature learning approach for anomaly detection in machine processes, based on a prototype few-shot learning technique that requires a limited number of labeled samples. The work is evaluated on a real-world scenario using the publicly available CNC Machining dataset. The proposed method outperforms the conventional prototypical network and the feature analysis shows a high generalization ability achieving an F1-score of 90.3%. The comparison with handcrafted features proves the robustness of the deep features and their invariance to data shifts across machines and time periods, which makes it a reliable method for sensory industrial applications. [ABSTRACT FROM AUTHOR]

Deep learning models have shown great potential in remote sensing (RS) image processing and analysis. Nevertheless, there are insufficient labeled samples to train deep networks, which seriously affects the performance of these models. To resolve this contradiction, we propose a generative self-supervised feature learning (S2FL) architecture for multimodal RS image land cover classification. Specifically, multiple complementary observed views are constructed from multimodal RS images, which are employed for following generative self-supervised learning (SSL). The proposed S2FL architecture is capable of extracting high-level meaningful feature representations from multiview data, and this process does not require any labeled information, providing a feasible solution to relieve the urgent need for annotated samples. The learned features are normalized and merged with corresponding spectral information to further improve the discriminative capability of feature representations, and we utilize these fused features for land cover classification. Compared with the existing supervised, semi-supervised, and self-supervised approaches, the proposed generative self-supervised model achieves superior performance in terms of feature learning and land cover classification, especially in the small sample classification case. [ABSTRACT FROM AUTHOR]

• Review on affective computing by single- and multi-modal analysis with benchmark databases. • In the retrospect of 19 relevantly latest reviews and 350+ researches papers by 2021. • Taxonomy of state-of-the-art methods considering ML-based or DL-based techniques. • Comparative summary of the properties and quantitative performance of key methods. • Discuss problems as well as some potential factors on affect recognition and analysis. Affective computing conjoins the research topics of emotion recognition and sentiment analysis, and can be realized with unimodal or multimodal data, consisting primarily of physical information (e.g., text, audio, and visual) and physiological signals (e.g., EEG and ECG). Physical-based affect recognition caters to more researchers due to the availability of multiple public databases, but it is challenging to reveal one's inner emotion hidden purposefully from facial expressions, audio tones, body gestures, etc. Physiological signals can generate more precise and reliable emotional results; yet, the difficulty in acquiring these signals hinders their practical application. Besides, by fusing physical information and physiological signals, useful features of emotional states can be obtained to enhance the performance of affective computing models. While existing reviews focus on one specific aspect of affective computing, we provide a systematical survey of important components: emotion models, databases, and recent advances. Firstly, we introduce two typical emotion models followed by five kinds of commonly used databases for affective computing. Next, we survey and taxonomize state-of-the-art unimodal affect recognition and multimodal affective analysis in terms of their detailed architectures and performances. Finally, we discuss some critical aspects of affective computing and its applications and conclude this review by pointing out some of the most promising future directions, such as the establishment of benchmark database and fusion strategies. The overarching goal of this systematic review is to help academic and industrial researchers understand the recent advances as well as new developments in this fast-paced, high-impact domain. [ABSTRACT FROM AUTHOR]

Impact load is the load that machines frequently experienced in engineering applications. Its time-history reconstruction and localization are crucial for structural health monitoring and reliability analysis. However, when identifying random impact loads, conventional inversion methods usually do not perform well because of complex formula derivation, infeasibility of nonlinear structure, and ill-posed problem. Deep learning methods have great ability of feature learning and nonlinear representation as well as comprehensive regularization mechanism. Therefore, a new feature learning-based method is proposed to conduct impact load reconstruction and localization. The proposed method mainly includes two parts. The first part is designed to reconstruct impact load, named convolutional-recurrent encoder–decoder neural network (ED-CRNN). The other part is constructed to localize impact load, called deep convolutional-recurrent neural network (DCRNN). The ED-CRNN utilizes the one-dimensional (1-D) convolutional encoder–decoder to obtain low-dimension feature representations of input signals. Two long short-term memory (LSTM) layers and a bidirectional LSTM (BiLSTM) layer are uniformly distributed in this network to learn the relationship between input features and the output load in time steps. The DCRNN is constructed mainly by two 1-D convolutional neural network (CNN) layers and two BiLSTM layers to learn high-hidden-level spatial as well as temporal features. The fully connected layers are placed at the end to localize an impact load. The effectiveness of the proposed method was demonstrated by two numerical studies and two experiments. The results reveal that the proposed method has the ability to accurately and quickly reconstruct and localize the impact load of complex assembled structure. Furthermore, the performance of the DCRNN is related to the number of sensors and the architecture of the network. Meanwhile, the strategy of alternating layout is proposed to reduce the number of training locations. [ABSTRACT FROM AUTHOR]

Due to the differences in radiation and geometric characteristics of optical and synthetic aperture radar (SAR) images, there is still a huge challenge for accurate matching. In this paper, we propose a patch-matching network (PM-Net) to improve the matching performance of optical and SAR images. First, a multi-level keypoints detector (MKD) with fused high-level and low-level features is presented to extract more robust keypoints from optical and SAR images. Second, we use a two-channel network structure to improve the image patch matching performance. Benefiting from this design, the proposed method can directly learn the similarity between optical and SAR image patches without manually designing features and descriptors. Finally, the MKD and two-channel net-work are trained separately on GL3D and QXS-SAROPT data sets, and the PM-Net is tested on multiple pairs of optical and SAR images. The experimental results demonstrate that the proposed method outperforms four advanced image matching networks on qualitative and quantitative assessments. The quantitative experiment results show that using our method correct matching points numbers are increased by more than 1.15 times, the value of F1-measure is raised by an average of 7.4% and the root mean squared error (RMSE) is reduced by more than 15.3%. The advantages of MKD and the two-channel network are also verified through ablation experiments. [ABSTRACT FROM AUTHOR]

It is common to have various clones from cross-seedlings or unintended planting by the farmers in a tea plantation. Since each tea clone has distinctive features such as quality, resistance to diseases, etc., visual inspections are usually conducted on the plantations to segment areas with different tea clones within the plantation to produce crops with consistent quality. However, this would be costly and time-consuming. In this work, we apply machine learning and develop an application to recognize tea clones automatically. We propose a convolutional variational autoencoder-based feature learning algorithm to produce robust features against data distortions. There are two main advantages of using this algorithm for feature learning. First, there is no need to design complex handcrafted features for classifications, usually conducted in machine learning. Second, the resulting features are more robust when tested with data taken from unideal conditions. The proposed method is evaluated using the original and the distorted image. Our proposed method achieves the best performance of 0.83 (83%) for the original image test, 0.75 (75%) for the gaussian blur image test, and 0.78 (78%) for the median blur image test. This is a much more robust result than VGGNet16, a popular supervised deep convolutional neural network. [ABSTRACT FROM AUTHOR]

Vibration signals are widely utilized for machinery fault diagnosis. However, the fault-related components (i.e., impulse) in vibration signals are often buried by strong background noises due to complex working conditions. Thus, it is challenging to directly extract discriminate features from vibration signals. In this article, a novel deep neural network (DNN), multiscale weighted morphological network (MWMNet), is proposed to extract impulses from vibration signals. First, a novel morphological layer is smoothly embedded in DNN as a signal processing layer to extract impulses and filter out the noise. Second, multiple branches with different structure element (SE) scales are employed to respectively extract impulses. An adaptive weighted fusion is utilized to enhance the scales that provide strong impulsive components. Third, a morphological operator, called average-hat transform, is adopted in MWMNet, and both positive and negative impulses can be extracted from vibration signals. The effectiveness of MWMNet is validated by the experiments of gearbox fault diagnosis and bearing fault diagnosis. The experimental results show that MWMNet can learn the fault-related features and filter out noise from vibration signals well. The comparison results illustrate that MWMNet has a better fault diagnosis performance than those DNNs, e.g., one-dimensional convolutional neural network, densely connected convolutional network, and residual network. [ABSTRACT FROM AUTHOR]

This article proposes a valve stiction detection strategy based on a convolutional neural network. Considering the commonly existing characteristics of industrial time-series signals, the strategy is developed to learn features on multiple timescales automatically. Unlike the traditional approaches using hand-crafted features, the proposed strategy can automatically learn representative features on the time-series data collected from industrial control loops. The strategy is composed of two complementary data conversion methods, a mixed feature learning stage and a fusion decision stage, and it has the following merits: 1) the interaction of different pairs of time series can be effectively captured; and 2) the whole process of feature learning is automatic, and no manual feature extraction is needed. The effectiveness of the proposed strategy is evaluated through the comprehensive data, including the International Stiction Data Base, and the real data collected from the real hardware experimental system and the industrial environment. Compared with four traditional methods and three deep-learning-based methods, the experimental results demonstrate that the proposed strategy outperforms the other methods. Besides performance evaluation, we give the implementation procedure of practical application of the proposed strategy and provide the detailed analysis from the perspective of the data conversion methods and the number of timescales. [ABSTRACT FROM AUTHOR]

CNN extracts the signal characteristics layer by layer through the local perception of convolution kernel, but the rotation speed and sampling frequency of the vibration signal of rotating equipment are not the same. Extracting different signal features with a fixed convolution kernel will affect the local feature perception and ultimately affect the learning effect and recognition accuracy. In order to solve this problem, the matching between the size of convolution kernel and the signal (rotation speed, sampling frequency) was optimized with the matching relation obtained. Through the study of this paper, the ability of extracting vibration features of CNN was improved, and the accuracy of vibration state recognition was finally improved to 98%. [ABSTRACT FROM AUTHOR]

Identifying accident patterns is one of the most vital research foci of driving analysis. Environmental or safety applications and the growing area of fleet management all benefit from accident detection contributions by minimizing the risk vehicles and drivers are subject to, improving their service and reducing overhead costs. Some solutions have been proposed in the past literature for automated accident detection that are mainly based on traffic data or external sensors. However, traffic data can be difficult to access, while external sensors can end up being difficult to set up and unreliable, depending on how they are used. Additionally, the scarcity of accident detection data has limited the type of approaches used in the past, leaving in particular, machine learning (ML) relatively unexplored. Thus, in this paper, we propose a ML framework for automated car accident detection based on mutimodal in-car sensors. Our work is a unique and innovative study on detecting real-world driving accidents by applying state-of-the-art feature extraction methods using basic sensors in cars. In total, five different feature extraction approaches, including techniques based on feature engineering and feature learning with deep learning are evaluated on the strategic highway research program (SHRP2) naturalistic driving study (NDS) crash data set. The main observations of this study are as follows: (1) CNN features with a SVM classifier obtain very promising results, outperforming all other tested approaches. (2) Feature engineering and feature learning approaches were finding different best performing features. Therefore, our fusion experiment indicates that these two feature sets can be efficiently combined. (3) Unsupervised feature extraction remarkably achieves a notable performance score. [ABSTRACT FROM AUTHOR]

• It is important to learn spatial-invariant feature representation for facial expression recognition which introduces no extra cost during inference. • Landmark is helpful for recognizing facial expressions with GCNs. • Global feature is indispensable for producing discriminative features. Despite the recent effort from computer vision community, facial expression recognition (FER) remains a largely unsolved problem. This is because the appearance of people's face undergoes dramatic changes due to changes in view angle, pose, illumination plus ambiguous facial expressions and low-quality facial images. In this work, we show the advantage of feature representation learning by dynamically graph message propagating subject to FER discriminative learning constraints and minimizing the distance of expression-agnostic transformed instance feature pairs. Specifically, we formulate a novel Harmonious Representation Learning (HRL) model for joint learning of landmark-guided graph message propagation, and spatially invariant feature learning using only generic matching metrics. Extensive comparative evaluations demonstrate the superiority of our proposed approach for FER over a variety of state-of-the-art methods on three major benchmark datasets including SFEW 2.0, RAF-DB, and CK+. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has been widely applied in process fault diagnosis due to its powerful feature extraction ability. A predominant property of these fault diagnosis models is to extract effective features from process signal. However, it is still difficult for them to construct the feature association relationship between input data. To solve these problems, this paper proposes a new graph neural network (GNN), pruning graph Convolutional network (PGCN), to perform feature learning based on the graph data. One dimensional process data are transformed into graph data by a graph construction method. A graph Convolutional network (GCN) is used to extract the features of process data. A pruning method of graph structure is proposed to effectively extract important information from process fault data. The feasibility and effectiveness of PGCN are verified on two benchmark processes, i.e., continuous stirred-tank reactor (CSTR) and fed-batch fermentation penicillin process (FBFP). The experimental results show that the performance of PGCN in feature extraction and process fault diagnosis is better than that of other typical methods, which provides a good possibility for the application of GCN in industrial process fault diagnosis. • A pruning graph Convolutional network (PGCN) is proposed to learn features from process variables. • The feature learning improves fault detection and diagnosis performance. • PGCN provides an effective way for process fault diagnosis. • The feasibility and effectiveness of PGCN are verified on two benchmark processes. [ABSTRACT FROM AUTHOR]

The past century has witnessed the grand success of brain imaging technologies, such as electroencephalography and magnetic resonance imaging, in probing cognitive states and pathological brain dynamics for neuroscience research and neurology practices. Human brain is "the most complex object in the universe," and brain imaging data (BID) are routinely of multiple/many attributes and highly non-stationary. These are determined by the nature of BID as the recordings of the evolving processes of the brain(s) under examination in various views. Driven by the increasingly high demands for precision, efficiency, and reliability in neuroscience and engineering tasks, dimensionality reduction has become a priority issue in BID analysis to handle the notoriously high dimensionality and large scale of big BID sets as well as the enormously complicated interdependencies among data elements. This has become particularly urgent and challenging in this big data era. Dimensionality reduction theories and methods manifest unrivaled potential in revealing key insights to BID via offering the low-dimensional/tiny representations/features, which may preserve critical characterizations of massive neuronal activities and brain functional and/or malfunctional states of interest. This study surveys the most salient work along this direction conforming to a 3-dimensional taxonomy with respect to (1) the scale of BID, of which the design with this consideration is important for the potential applications; (2) the order of BID, in which a higher order denotes more BID attributes manipulatable by the method; and (3) linearity, in which the method's degree of linearity largely determines the "fidelity" in BID exploration. This study defines criteria for qualitative evaluations of these works in terms of effectiveness, interpretability, efficiency, and scalability. The classifications and evaluations based on the taxonomy provide comprehensive guides to (1) how existing research and development efforts are distributed and (2) their performance, features, and potential in influential applications especially when involving big data. In the end, this study crystallizes the open technical issues and proposes research challenges that must be solved to enable further researches in this area of great potential. [ABSTRACT FROM AUTHOR]

Progress in the digitization of cultural assets leads to online databases that become too large for a human to analyze. Moreover, some analyses might be challenging, even for experts. In this paper, we explore two applications of computer vision to analyze historical data: watermark recognition and one-shot repeated pattern detection in artwork collections. Both problems present computer vision challenges which we believe to be representative of the ones encountered in cultural heritage applications: limited supervision is available, the tasks are fine-grained recognition, and the data comes in several different modalities. Both applications are also highly practical, as recognizing watermarks makes it possible to date and locate documents, while detecting repeated patterns allows exploring visual links between artworks. We demonstrate on both tasks the benefits of relying on deep mid-level features. More precisely, we define an image similarity score based on geometric verification of mid-level features and show how spatial consistency can be used to fine-tune out-of-the-box features for the target dataset with weak or no supervision. This paper relates and extends our previous works (Shen et al. in Discovering visual patterns in art collections with spatially-consistent feature learning, 2019; Shen et al. in Large-scale historical watermark recognition dataset and a new consistency-based approach, 2020). Our code and data are available at http://imagine.enpc.fr/~shenx/HisImgAnalysis/. [ABSTRACT FROM AUTHOR]

To solve the problem of feature representation of complex acoustic environment in acoustic scene classification task, an optimization algorithm of joint training feature extraction and classification model is proposed. In order to learn more discriminative and supervised features, non-negative matrix factorization is combined with convolution neural network training, and the loss value of network is used to realize feature extraction and network parameters updating. The logarithmic spectrogram is extracted from the TUT2017 dataset as the basic feature. And the deep convolutional neural network is built for experimental verification. The simulation results show that the recognition accuracy of the proposed algorithm is improved by 3. 9 % compared with that before optimization, and is superior to the other two commonly used acoustic features, which proves that the algorithm can effectively improve the overall classification effect. [ABSTRACT FROM AUTHOR]

In modern societies, handwritten identification is a great practical need and challenge for forensic sciences. Moreover, few research studies have focused on automatic offline handwritten document analysis. Also, there are very few standard datasets on handwritten document identification. In addition, handwritten documents lose their nature over time due to the spread and drying of ink. From this standpoint, the present study presents an offline writer identification system (in presence of the uncertainties such as different experimental conditions and environmental noises for more realistic assumptions) that is a critical need in forensic studies. For this purpose, a comprehensive right-to-left dataset is developed and devised by gathering data from 62 participants at different time intervals under different experimental conditions. This dataset is designed based on American Society for Testing and Materials (ASTM) standards. A Deep Convolutional Neural Network (DCNN) model based on modified pre-trained networks is designed and developed to extract features from raw data hierarchically. The proposed DCNN model is tested and investigated not only on the designed dataset but also on different datasets. One notable advantage of the present study is that it has made use of heterogeneous data. Another remarkable aspect of the proposed study is that the proposed DCNN model is independent of any specific languages which can be applied on various languages. The results of the study indicate that the proposed DCNN model can learn features hierarchically from the handwriting raw data and achieve higher accuracy than other comparative methods. [ABSTRACT FROM AUTHOR]

Performing clustering analysis on a large amount of historical trajectory data can obtain information such as frequent flight patterns of aircraft and air traffic flow distribution, which can provide a reference for the revision of standard flight procedures and the optimization of the division of airspace sectors. At present, most trajectory clustering uses a single clustering algorithm. When other processing remains unchanged, it is difficult to improve the clustering effect by using a single clustering method. Therefore, this paper proposes a trajectory clustering ensemble method based on a similarity matrix. Firstly, a stacked autoencoder is used to learn a small number of features that are sufficiently representative of the trajectory and used as the input to the subsequent clustering algorithm. Secondly, each basis cluster is used to cluster the data set, and then a consistent similarity matrix is obtained by using the clustering results of each basis cluster. On this basis, using the deformation of the matrix as the distance matrix between trajectories, the agglomerative hierarchical clustering algorithm is used to ensemble the results of each basis cluster. Taking the Nanjing Lukou Airport terminal area as an example, the experimental results show that integrating multiple basis clusters eliminates the inherent randomness of a single clustering algorithm, and the trajectory clustering results are more robust. [ABSTRACT FROM AUTHOR]

Pedestrian detection in the wild remains a challenging problem especially for scenes containing serious occlusion. In this paper, we propose a novel feature learning method in the deep learning framework, referred to as Feature Calibration Network (FC-Net), to adaptively detect pedestrians under various occlusions. FC-Net is based on the observation that the visible parts of pedestrians are selective and decisive for detection, and is implemented as a self-paced feature learning framework with a self-activation (SA) module and a feature calibration (FC) module. In a new self-activated manner, FC-Net learns features which highlight the visible parts and suppress the occluded parts of pedestrians. The SA module estimates pedestrian activation maps by reusing classifier weights, without any additional parameter involved, therefore resulting in an extremely parsimony model to reinforce the semantics of features, while the FC module calibrates the convolutional features for adaptive pedestrian representation in both pixel-wise and region-based ways. Experiments on CityPersons and Caltech datasets demonstrate that FC-Net improves detection performance on occluded pedestrians up to 10% while maintaining excellent performance on non-occluded instances. [ABSTRACT FROM AUTHOR]

Deep learning has revolutionized the remote sensing image processing techniques over the past few years. Nevertheless, annotating high-quality samples is difficult and time-consuming, which limits the performance of deep neural networks because of insufficient supervision information. Aiming to solve this contradiction, we investigate the multimodal self-supervised learning (MultiSSL) paradigm for pre-training and classification of remote sensing image. Specifically, the proposed self-supervised feature learning model consists of asymmetric encoder–decoder structure, in which deep unified encoder learns high-level key information characterizing multimodal remote sensing data and task-specific lightweight decoders are developed to reconstruct original data. To further enhance feature extraction capability, the cross-attention layers are utilized to exchange information contained in heterogeneous characteristics, thus learning more complementary information from multimodal remote sensing data. In fine-tuning stage, the pre-trained encoder and cross-attention layer serve as feature extractor, and leaned characteristics are combined with corresponding spectral information for land cover classification through a lightweight classifier. The self-supervised pre-training model can learn high-level key features from unlabeled samples, thereby utilizing the feature extraction capability of deep neural networks while reducing their dependence on annotated samples. Compared with existing classification paradigms, the proposed multimodal self-supervised pre-training and fine-tuning scheme achieves superior performance for remote sensing image land cover classification. • We bring forward self-supervised pre-training for multimodal remote sensing image. • The multimodal self-supervised learning model is investigated for feature learning. • Cross attention module exchanges heterogeneous features between different modalities. • Learned multimodal features are fused with spectral information for classification. • The proposed method achieves superior performance on multimodal benchmark datasets. [ABSTRACT FROM AUTHOR]

With the rapid growth of big data analysis, graphs have become an important data structure in relationship extraction and learning. However, the complexity of graph structure increases the difficulty of downstream learning tasks, e.g., graph classification. For example, some traditional graph classification methods rely on the attributes of nodes and edges; the attributes may be incomplete or missing in large-scale graph datasets, e.g., online social networks and knowledge graphs. Focusing on the node-level features overlooks the global characteristics, leading to decreased classification accuracy. This paper proposes a novel graph classification method based on the subgraph-level feature the high-difference-frequency subgraph. We measure the difference in subgraph appearing frequency and use the high difference frequency subgraphs as features in graph learning and classification. The experiments demonstrate that the proposed method outperforms the state-of-the-art graph classification methods. [ABSTRACT FROM AUTHOR]

Many chemical and biochemical systems can be intuitively modeled using networks. Due to the size and complexity of many biochemical networks, we require tools for efficient network analysis. Of particular interest are techniques that embed network vertices into vector spaces while preserving important properties of the original graph. In this article, we {introduce a new method for generating low-dimensional node embeddings for directed graphs, using random walk sampling methods for feature learning on networks. Additionally, we demonstrate the usefulness of this method for identifying transition states of stochastic chemical reacting systems.} Network representations of chemical systems are typically given by weighted directed graphs, and are often complex and high dimensional. In order to deal with networks representing these chemical systems, therefore, we modified objective functions adopted in existing random walk based network embedding methods to handle directed graphs and neighbors of different degrees. Through optimization via gradient ascent, we embed the weighted graph vertices into a low-dimensional vector space Rd while preserving the neighborhood of each node. These embeddings may then be used to detect relationships between nodes and study the structure of the original network. We then demonstrate the effectiveness of our method on dimension reduction through several examples regarding identification of transition states of chemical reactions, especially for entropic systems.

Medical images play a vital role in medical analysis by providing crucial information about patients' pathological conditions. However, the quality of these images can be compromised by many factors, such as limited resolution of the instruments, artifacts caused by movements, and the complexity of the scanned areas. As a result, low-resolution (LR) images cannot provide sufficient information for diagnosis. To address this issue, researchers have attempted to apply image super-resolution (SR) techniques to restore the high-resolution (HR) images from their LR counterparts. However, these techniques are designed for generic images, and thus suffer from many challenges unique to medical images. An obvious one is the diversity of the scanned objects; for example, the organs, tissues, and vessels typically appear in different sizes and shapes, and are thus hard to restore with standard convolution neural networks (CNNs). In this paper, we develop a dynamic-local learning framework to capture the details of these diverse areas, consisting of deformable convolutions with adjustable kernel shapes. Moreover, the global information between the tissues and organs is vital for medical diagnosis. To preserve global information, we propose pixel–pixel and patch–patch global learning using a non-local mechanism and a vision transformer (ViT), respectively. The result is a novel CNN-ViT neural network with Local-to-Global feature learning for medical image SR, referred to as LGSR, which can accurately restore both local details and global information. We evaluate our method on six public datasets and one large-scale private dataset, which include five different types of medical images (i.e. , Ultrasound, OCT, Endoscope, CT, and MRI images). Experiments show that the proposed method achieves superior PSNR/SSIM and visual performance than the state of the arts with competitive computational costs, measured in network parameters, runtime, and FLOPs. What is more, the experiment conducted on OCT image segmentation for the downstream task demonstrates a significantly positive performance effect of LGSR. • We proposed a novel CNN-ViT model, LGSR, for multi-modality medical image SR. • We defined a local-to-global learning framework to restore information thoroughly. • LGSR costs fewer parameters than CNN SOTAs, and is faster than CNN-ViT SOTAs. [ABSTRACT FROM AUTHOR]

Identifying and verifying the identity of people based on scanned images of handwritten documents is an applicable biometric modality with applications in forensic and historic document investigation, and it is an important study area within the research field of behavioral biometrics. Despite this, there are few studies in this field. Furthermore, there are very few standard datasets for identifying and verify handwritten documents. Also, handwritten documents lose their character during time because of ink spread and drying. Therefore, it is necessary to provide a method that can identify and verify handwritten documents under various uncertainties. In this study, a text-independent writer identification and verification model in offline state under different experimental conditions is developed using a combination of Deep Type-2 Fuzzy architecture and Transfer Learning networks (DT2F-TLNet). So, a right-to-left dataset has been collected. The proposed DT2F-TLNet model is validated using both the designed dataset and other benchmark datasets. The proposed model is distinguished by the fact that it is developed to be independent of the textual content of the handwritten cases and can be used for various languages. The study's findings show that the developed DT2F-TLNet model can learn properties from heterogeneous handwriting data and results in higher accuracy than other comparable approaches. [ABSTRACT FROM AUTHOR]

Microstructure is key to controlling and understanding the properties of materials, but traditional approaches to describing microstructure capture only a small number of features. We require more complete descriptors of microstructure to enable data-centric approaches to materials discovery, to allow efficient storage of microstructural data and to assist in quality control in metals processing. The concept of microstructural fingerprinting , using machine learning (ML) to develop quantitative, low-dimensional descriptors of microstructures, has recently attracted significant attention. However, it is difficult to interpret conclusions drawn by ML algorithms, which are often referred to as "black boxes". For example, convolutional neural networks (CNNs) can be trained to make predictions about a material from a set of microstructural image data, but the feature space that is learned is often used uncritically and adopted without any validation. Here we explore the use of variational autoencoders (VAEs), comprising a pair of CNNs, which can be trained to produce microstructural fingerprints in a continuous latent space. The VAE architecture also permits the reconstruction of images from fingerprints, allowing us to explore how key features of microstructure are encoded in the latent space of fingerprints. We develop a VAE architecture based on ResNet18 and train it on two classes of Ti-6Al-4V optical micrographs (bimodal and lamellar) as an example of an industrially important alloy where microstructural control is critical to performance. The latent/feature space of fingerprints learned by the VAE is explored in several ways, including by supplying interpolated and randomly perturbed fingerprints to the trained decoder and via dimensionality reduction to explore the distribution and correlation of microstructural features within the latent space of fingerprints. We demonstrate that the fingerprints generated via the trained VAE exhibit smooth, interpolable behaviour with stability to local perturbations, supporting their suitability as general purpose descriptors for microstructure. The analysis of computational results uncover that key properties of the microstructures (volume fraction and grain size) are strongly correlated with position in the encoded feature space, supporting the use of VAE fingerprints for quantitative exploration of process–structure–property relationships. [Display omitted] • VAE based on ResNet is able to produce accurate reconstructions of microstructures. • PCA provides useful insight into the boundaries of validity in the latent space. • t -SNE applied to latent space to visualise morphological feature distributions. • VAE fingerprints shown to correlate with morphological features via regression. [ABSTRACT FROM AUTHOR]

Reward learning enables robots to learn adaptable behaviors from human input. Traditional methods model the reward as a linear function of hand-crafted features, but that requires specifying all the relevant features a priori, which is impossible for real-world tasks. To get around this issue, recent deep Inverse Reinforcement Learning (IRL) methods learn rewards directly from the raw state but this is challenging because the robot has to implicitly learn the features that are important and how to combine them, simultaneously. Instead, we propose a divide-and-conquer approach: focus human input specifically on learning the features separately, and only then learn how to combine them into a reward. We introduce a novel type of human input for teaching features and an algorithm that utilizes it to learn complex features from the raw state space. The robot can then learn how to combine them into a reward using demonstrations, corrections, or other reward learning frameworks. We demonstrate our method in settings where all features have to be learned from scratch, as well as where some of the features are known. By first focusing human input specifically on the feature(s), our method decreases sample complexity and improves generalization of the learned reward over a deep IRL baseline. We show this in experiments with a physical 7-DoF robot manipulator, and in a user study conducted in a simulated environment. [ABSTRACT FROM AUTHOR]

In real-world complex networks, understanding the dynamics of their evolution has been of great interest to the scientific community. Predicting non-existent but probable links is an essential task of social network analysis as the addition or removal of the links over time leads to the network evolution. In a network, links can be categorized as intra-community links if both end nodes of the link belong to the same community, otherwise inter-community links. The existing link-prediction methods have mainly focused on achieving high accuracy for intra-community link prediction. In this work, we propose a network embedding method, called NodeSim, which captures both similarities between the nodes and the community structure while learning the low-dimensional representation of the network. The embedding is learned using the proposed NodeSim random walk, which efficiently explores the diverse neighborhood while keeping the more similar nodes closer in the context of the node. We verify the efficacy of the proposed embedding method over state-of-the-art methods using diverse link prediction. We propose a machine learning model for link prediction that considers both the nodes' embedding and their community information to predict the link between two given nodes. Extensive experimental results on several real-world networks demonstrate the effectiveness of the proposed method for both inter and intra-community link prediction. [ABSTRACT FROM AUTHOR]

In this study, we proposed a machine learning (ML) method to optimize the comprehensive performance of shape memory epoxy polymers (SMEPs) based on experimental data as samples. Firstly, a series of SMEPs specimens were prepared, and their properties were evaluated respectively by testing four indexes including the glass transition temperature (Tg), bending strength, strain fixation rate (Rf), and strain recovery rate (Rr). Subsequently, ML used these experimental data as samples for feature learning to investigate the influence of each component on these properties. The results indicated that methyltetrahydrophthalic anhydride was more favorable to Tg and bending strength than methylhexahydrophthalic anhydride (MHHPA) as a curing agent. However, a certain amount of MHHPA must be included in the system to guarantee a higher Rf and Rr. Moreover, the right amount of bisphenol A cyanate ester in the system improved the comprehensive properties of SMEPs, especially the shape memory effect. Finally, a SMEPs system with superior properties was acquired through the optimization of four indexes of Tg, bending strength, Rf and Rr. Therefore, this study shows that ML methods can also be used to investigate SMEPs that require more specific excellent performance. [ABSTRACT FROM AUTHOR]

In real-world applications such as emotion recognition from recorded brain activity, data are captured from electrodes over time. These signals constitute a multidimensional time series. In this article, Echo State Network (ESN), a recurrent neural network with great success in time series prediction and classification, is optimized with different neural plasticity rules for classification of emotions based on electroencephalogram (EEG) time series. The developed network could automatically extract valid features from EEG signals. We use the filtered signals as the network input and do not take any feature extraction methods. Evaluated on two well-known benchmarks, the DEAP dataset, and the SEED dataset, the performance of the ESN with intrinsic plasticity greatly outperforms the feature-based methods and shows certain advantages compared with other existing methods. Thus, the proposed network can form a more complete and efficient representation, whilst retaining the advantages such as faster learning speed and more reliable performance. [ABSTRACT FROM AUTHOR]

Using evolutionary computation algorithms to solve multiple tasks with knowledge sharing is a promising approach. Image feature learning can be considered as a multitask learning problem because different tasks may have a similar feature space. Genetic programming (GP) has been successfully applied to image feature learning for classification. However, most of the existing GP methods solve one task, independently, using sufficient training data. No multitask GP method has been developed for image feature learning. Therefore, this article develops a multitask GP approach to image feature learning for classification with limited training data. Owing to the flexible representation of GP, a new knowledge sharing mechanism based on a new individual representation is developed to allow GP to automatically learn what to share across two tasks and to improve its learning performance. The shared knowledge is encoded as a common tree, which can represent the common/general features of two tasks. With the new individual representation, each task is solved using the features extracted from a common tree and a task-specific tree representing task-specific features. To find the best common and task-specific trees, a new evolutionary search process and fitness functions are developed. The performance of the new approach is examined on six multitask learning problems of 12 image classification datasets with limited training data and compared with 17 competitive methods. The experimental results show that the new approach outperforms these comparison methods in almost all the comparisons. Further analysis reveals that the new approach learns simple yet effective common trees with high effectiveness and transferability. [ABSTRACT FROM AUTHOR]

Earth scientists increasingly deal with 'big data'. For spatial interpolation tasks, variants of kriging have long been regarded as the established geostatistical methods. However, kriging and its variants (such as regression kriging, in which auxiliary variables or derivatives of these are included as covariates) are relatively restrictive models and lack capabilities provided by deep neural networks. Principal among these is feature learning: the ability to learn filters to recognise task-relevant patterns in gridded data such as images. Here, we demonstrate the power of feature learning in a geostatistical context by showing how deep neural networks can automatically learn the complex high-order patterns by which point-sampled target variables relate to gridded auxiliary variables (such as those provided by remote sensing) and in doing so produce detailed maps. In order to cater for the needs of decision makers who require well-calibrated probabilities, we also demonstrate how both aleatoric and epistemic uncertainty can be quantified in our deep learning approach via a Bayesian approximation known as Monte Carlo dropout. In our example, we produce a national-scale probabilistic geochemical map from point-sampled observations with auxiliary data provided by a terrain elevation grid. By combining location information with automatically learned terrain derivatives, our deep learning approach achieves an excellent coefficient of determination ( R 2 = 0.74 ) and near-perfect probabilistic calibration on held-out test data. Our results indicate the suitability of Bayesian deep learning and its feature-learning capabilities for large-scale geostatistical applications where uncertainty matters. [ABSTRACT FROM AUTHOR]

Conventional biometric modalities, such as the face, fingerprint, and iris, are vulnerable against imitation and circumvention. Accordingly, secure biometric modalities with cancelable properties are needed for personal identification, especially in smart healthcare applications. Here we developed a person identification model using high-density surface electromyography (HD-sEMG) as biometric traits. In this model, the HD-sEMG biometric templates are cancelable and could be customized by the users through finger isometric contractions. A deep feature learning approach, implemented by convolutional neural networks (CNNs) is used to capture user-specific patterns from HD-sEMG signals and make identification decisions. This model has been validated on twenty-two subjects, with training and testing data acquired from two different days. The rank-1 identification accuracy and equal error rate for 44 identities (22 subjects × 2 accounts) can reach 87.23% and 4.66%, respectively. The cross-day identification accuracy of the proposed model is higher than the results of previous methods reported in the literature. The usability and efficiency of the proposed model are also investigated, indicating its potentials for practical applications. [ABSTRACT FROM AUTHOR]