Search

Abstract The railway track fasteners play a critical role in fixing the track on the ballast bed. Achieving full automation of the fastener defect detection is significant in terms of ensuring track safety, and reducing maintains cost. In this paper, innovative and intelligent methods using image processing technologies and deep learning networks are proposed. In the first part, the traditional fastener positioning method based on image processing is reconsidered. In addition, a novel fastener defect detection and identification method using Dense-SIFT features is proposed which can achieve a better performance than the methods available in the literature. In the second part, VGG16 is trained for fastener defect detection and recognition. The result demonstrates that it is possible to carry out the defect detection of fasteners with CNN. Finally, Faster R-CNN is used for fastener defect detection to advance detection rate and efficiency. The fastener positioning and recognition can be carried out simultaneously. The time for the defect detection and classification is only one-tenth of the other methods mentioned above. Highlights • The Dense-SIFT, spatial pyramid decomposition and BOVW techniques are applied to the classification and this method achieves classification accuracy of 99.26%. • The DCNN method uses only one network for fastener defects detection and achieves a sound classification accuracy of 97.14%. • Faster R-CNN based method realizes the positioning and classification of the fasteners simultaneously and the time spent by Faster R-CNN is only about 10% of the time spent by the other methods. [ABSTRACT FROM AUTHOR]

Abstract Recently, automatic feature extraction and selection from unlabeled images that contain irrelevant patterns have been a proceeding interest. In this paper, an enhanced high-order Boltzmann machine is designed to promote the capacity of feature extraction and selection in a unified context. First, gating mechanism is employed for feature selection in comparison with conventional approaches. Then, two sets of hidden variables that the one set is real-valued latent variables and the other is spike latent variables are introduced to model the covariance structure of local patches, which can boost the abilities of feature learning and feature selection in turn. Simultaneously, the proposed model can infer in parallel via easy block Gibbs sampling without much training difficulty. Last, several extensions of the proposed model are developed to cope with different scenes. The massive performances obtained from various visual tasks have demonstrated that the proposed model can reach the highly improved performances over several currently excellent methods. [ABSTRACT FROM AUTHOR]

Abstract This paper concerns the issue of monitoring elderly behavior in the context of ambient assisted living (AAL). Under the framework of online daily habit modeling (ODHM), we employ the emergent representation for activities of daily living (ADLs) with position-based stigmergy, and then combine it with convolution neural networks (CNNs) to accomplish the tasks of recognizing ADLs. In addition, we propose a new paradigm of activity summarization with the robustness to break interruptions. Radio tomographic imaging (RTI) is promoted as a simple yet flexible way of facilitating the required position-based stigmergy. Such position-based AAL systems can benefit the advantages of having no need any sophisticated domain models in analyzing and understanding ADLs while no burden training is involved in ODHM. Moreover, the emergent based data aggregation and deep learning of CNN together allow the recognition of ADLs at a fine-grained level, which contributes to the performance improvement of ODHM. Experimental results demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

Malware writers are usually focused on those platforms which are most used among common users, with the aim of attacking as many devices as possible. Due to this reason, Android has been heavily attacked for years. Efforts dedicated to combat Android malware are mainly concentrated on detection, in order to prevent malicious software to be installed in a target device. However, it is equally important to put effort into an automatic classification of the type, or family, of a malware sample, in order to establish which actions are necessary to mitigate the damage caused. In this paper, we present CANDYMAN, a tool that classifies Android malware families by combining dynamic analysis and Markov chains. A dynamic analysis process allows to extract representative information of a malware sample, in form of a sequence of states, while a Markov chain allows to model the transition probabilities between the states of the sequence, which will be used as features in the classification process. The space of features built is used to train classical Machine Learning, including methods for imbalanced learning, and Deep Learning algorithms, over a dataset of malware samples from different families, in order to evaluate the proposed method. Using a collection of 5,560 malware samples grouped into 179 different families (extracted from the Drebin dataset), and once made a selection based on a minimum number of relevant and valid samples, a final set of 4,442 samples grouped into 24 different malware families was used. The experimental results indicate a precision performance of 81.8% over this dataset. [ABSTRACT FROM AUTHOR]

Obstructive sleep apnea (OSA) is a common type of sleep-related breathing disorder, and polysomnography (PSG) remains the gold standard for its diagnosis. However, it takes a significant amount of time to perform PSG in a well-equipped laboratory, and patients typically have to wait a long time for a PSG test. In view of this, over recent years portable and even wearable tools for OSA classification have been developed as a low-cost and easy-to-use alternative to PSG. In this paper, a deep neural network (DNN)-based model was developed to classify OSA severity using peripheral oxygen saturation (SpO 2) signals; it showed the following advantages. First, the presented model takes unsegmented SpO 2 signals recorded overnight as its input, and OSA severity is then classified as one of four levels as the output. Consequently, there is obviously no need to label segmented signals, and the tremendous amount of effort spent on signal segmentation and then annotation can be completely saved. Second, a high generalization ability is provided since the largest amount of data were used to test the model. This feature gives the model an improved reliability for clinical use. Notably, the outperformance of this work is highlighted in a two-level classification case (with a cutoff apnea–hypopnea index of 5), where the accuracy and the sensitivity increased to above 91% and 95%, respectively. [Display omitted] • A novel method is proposed to classify OSA severity using unsegmented SpO2 signals. • The time and effort spent on signal segmentation and annotation can be fully saved. • A high generalization ability is provided in the presented model. • This work significantly outperformed counterparts in terms of accuracy. [ABSTRACT FROM AUTHOR]

Individuals in any country are badly impacted both economically and physically whenever an epidemic of infectious illnesses breaks out. A novel coronavirus strain was responsible for the outbreak of the coronavirus sickness in 2019. Corona Virus Disease 2019 (COVID-19) is the name that the World Health Organization (WHO) officially gave to the pneumonia that was caused by the novel coronavirus on February 11, 2020. The use of models that are informed by machine learning is currently a major focus of study in the field of improved forecasting. By displaying annual trends, forecasting models can be of use in performing impact assessments of potential outcomes. In this paper, proposed forecast models consisting of time series models such as long short-term memory (LSTM), bidirectional long short-term memory (Bi-LSTM), generalized regression unit (GRU), and dense-LSTM have been evaluated for time series prediction of confirmed cases, deaths, and recoveries in 12 major countries that have been affected by COVID-19. Tensorflow1.0 was used for programming. Indices known as mean absolute error (MAE), root means square error (RMSE), Median Absolute Error (MEDAE) and r2 score are utilized in the process of evaluating the performance of models. We presented various ways to time-series forecasting by making use of LSTM models (LSTM, BiLSTM), and we compared these proposed methods to other machine learning models to evaluate the performance of the models. Our study suggests that LSTM based models are among the most advanced models to forecast time series data. [ABSTRACT FROM AUTHOR]

When addressing product quality standards in manufacturing lines, a critical issue is the identification of the parameters that define the quality of the final product and their tracking. The problem of process control under inconsistent working condition of an automatic machinery, i.e. when some parameters are highly variable, is still quite unexplored in literature. This objective becomes even more challenging when the most important process variables are not directly measurable. This paper demonstrates that it is possible to achieve quality control by coupling a soft sensor, whose predictive model is a neural network, with an anomaly detector. The methodology has been applied to automatic machinery placed in a manufacturing line, where high variability in production rate has an important effect on the measured physical variables. This makes automated and accurate quality control difficult, due to the fact that in this test case the data collected are accelerometers signals, extremely sensible to variation in machine productivity by definition. It is shown that this approach outperforms many other classification methods (Support Vector Machines, Ensemble Bagged Tree, Discriminant Analysis, K-nearest neighbours and the direct application of a Neural Network) proposed in the past, achieving satisfactory results evaluated on the basis of four metrics (Accuracy, precision, recall and F 1 -score), even if anomalous data have been collected in a limited number of machine's working points. In particular, an accuracy over 92% has been reached also for production rates where only nominal conditions are collected. This procedure exceeds the direct training of a neural network (accuracy of 57.6% at new production rates), as well as the application of shallow methods based on the extraction of dimensionless features (around 35% in accuracy at new production rates). • Quality control is arduous when it is not possible to measure process variables. • Hard to build robust soft sensor with system working under inconsistent conditions. • A neural network can be trained at specific machine working points. • Anomaly detection is achieved by deep learning and a probabilistic model. • Accuracy higher than 92% for each production rate tested. [ABSTRACT FROM AUTHOR]

In data-driven tire defect detection, scarce defect samples and the poor transferability of models make the defect detection a time-consuming and expensive process. In recent years, domain adaptation methods have been applied to solve the cross-domain defect detection problems. However, previous works mainly assume that all regions in the image are equally transferable and align global features of the source and target domains with the same weight, which will fail for such a complex scenario. To this end, with the consideration of variability of transferability in different image regions, this paper presents Transferable Swin Transformer (TST), focusing the proposed method on region-based adversarial alignment and the semantic-based subdomain adaptation, simultaneously. Meanwhile, the subdomain adaptation benefits from the designed region-level weighting mechanism. The above strategies enable TST to learn both transferable and discriminative features in defect detection under domain shifts. Six transfer tasks constructed by different tire quality inspection X-ray image machines and a benchmark dataset are processed to demonstrate the effectiveness and generalization of the proposed method. • A novel Transferable Swin Transformer is proposed for transfer tire defect detection. • A region-level weighting mechanism is designed for the subdomain adaptation. • The proposed method performs well on the tire defect dataset and a benchmark dataset. [ABSTRACT FROM AUTHOR]

Existing fine-tuning methods mainly leverage the discriminative knowledge and discard the intrinsic structure of data. In this paper, we propose a novel framework Momentum Contrastive Bi-Tuning (MCBiT) for intelligent diagnosis of rotating machinery, which can fully exploit both the discriminative knowledge of labels and the intrinsic structure of target data in a boosting fine-tuning way. One-dimensional vibration signals are transformed by Gramian Angular Difference Field (GADF) and fed into MCBiT, which enhances the conventional fine-tuning by integrating two branches on the ImageNet-pretrained backbone: a classifier with an instance-contrastive cross-entropy loss to better exploit label knowledge; and a projector with a categorical contrastive learning loss to mining the intrinsic structure of data. Our proposed approach outperforms state-of-the-art methods on six publicly available rotating machinery fault diagnosis datasets and our experimental-collected dataset at different data scales. The promising performance of our proposed MCBiT contributes toward more practical data-driven approaches that can realize timely deployment under challenging real-world environments. [ABSTRACT FROM AUTHOR]

Automatic seismic phase detection and phase arrival time picking are always fundamental tasks in seismology. With the emergence of machine learning, and deep learning in particular, a number of neural network architectures are being used to explore the correlations between seismic waveforms and hidden information of interest. In this paper, we present a sequential attention (SEA) module that can learn both local and long-distance corrections to effectively extract sequential features. This module is lightweight and plug-in-play. Based on the SEA module, a fully convolutional neural network with sequential attention mechanism was constructed to detect seismic events and determine the arrival time of phases. Further experiments on STEAD dataset, in comparison with existing methods, show that our proposed model outperforms other algorithms with a higher F1-score (100% for seismic detection, 98.8% for P arrival picking, and 89.0% for S arrival picking) and fewer residual errors. Comparatively, our model has a quarter of the model parameters and computational complexity as well as a 36.7% reduction in processing time compared to the current state-of-the-art method. This demonstrates that the proposed architecture is effective for seismic sequential learning tasks and, as a result, can serve as an improved alternative backbone. [ABSTRACT FROM AUTHOR]

Automatic modulation classification (AMC) is an essential and indispensable topic in the development of cognitive radios. It is the cornerstone of adaptive modulation and demodulation capabilities to perceive and understand surrounding environments and make corresponding decisions. In this paper, we propose a priori regularization method in deep learning (DL-PR) for guiding loss optimization during model training process. The regularization factor designed by the combination of inter-class confrontation factor, global and dimensional divergence can help increase the inter-class distance and reduce the intra-class distance of samples. While preserving the original information of received signals as much as possible, it makes full use of the prior knowledge in the signal transmission process and ultimately helps deep learning models to be well generalized on signals with various signal-to-noise ratios (SNRs). As far as we know, this is the first attempt to regularize deep learning models based on SNR distribution of samples to improve AMC accuracy. Moreover, it can be proved that priori regularization can be interpreted as implicit data augmentation and model ensemble methods. By comparing with a series of state-of-the-art AMC methods and different regularization techniques on the public dataset RadioML 2016.10a, experimental results of multiple deep learning models illustrate the superiority of DL-PR, including CNN with accuracy of 62.6% and inference time of 0.82 ms per signal, LSTM with 61.8% and 0.87 ms, and hybrid CNN–LSTM with 64.2% and 0.94 ms. In practical applications, DL-PR can be also easily applied to complex environments due to its robustness to hyper-parameters and SNR estimation. [ABSTRACT FROM AUTHOR]

Missions involving Unmanned Aerial Vehicle usually consist of reaching a set of regions, performing some actions in each region, and returning to a determined depot after all the regions have been successfully visited or before the fuel/battery is totally consumed. Hence, planning a route becomes an important task for many applications, especially if a team of Unmanned Aerial Vehicles is considered. From this team, coordination and cooperation are expected to optimize results of the mission. In this paper, a system for managing multiple cooperative Unmanned Aerial Vehicles is presented. This system divides the routing problem into two stages: initial planning and routing solving. Initial planning is a first step where the regions to be visited are grouped in multiple clusters according to a distance criterion, with each cluster being assigned to each of the Unmanned Aerial Vehicles. Routing solving computes the best route for every agent considering the clusters of the initial planning and a variant of the Orienteering Problem. This variant introduces the concept of shared regions, allowing an Unmanned Aerial Vehicle to visit regions from other clusters and compensating for the suboptimal region clustering of the previous stage. The Orienteering Problem with shared regions is solved using the deep learning architecture Transformer along with a deep reinforcement learning framework. This architecture is able to obtain high-quality solutions much faster than conventional optimization approaches. Extensive results and comparison with other Combinatorial Optimization algorithms, including cooperative and non-cooperative scenarios, have been performed to show the benefits of the proposed solution. [ABSTRACT FROM AUTHOR]

Floods are natural phenomena that invade different parts of the globe annually, leaving severe adverse effects on the natural landscape, particularly in areas where humans practice large-scale urban development activities. Inadequate corporate strategic stormwater management or maintenance plans make the issue even worse. Therefore, efforts have been consolidated to confront flood hazards by incorporating precautious and proactive strategies toward better forecasting and preventing flood disasters or even mitigating the damage if it occurs. Further, there is inadequacy in incorporating computational intelligence techniques to detect flood susceptibility in road networks. This paper proposes a data-driven flood risk areas detection model incorporating various ensemble and generic machine learning techniques as well as autoML tools. A systematic and iterative approach is followed to examine different solutions to tackle the data balancing problem, including oversampling and undersampling techniques. Also, hyperparameter optimisation is undertaken using both grid search and genetic algorithms, and the models' performance using each designated strategy is reported and assessed. The empirical results from the experiments indicate that the optimised ensembled Extremely Randomized Trees classifier proves an ability to perform well on the given dataset with an averaged ROC AUC value of 90% and outperforms several state-of-the-art generic, ensembled, and autoML classifiers in all evaluation measures. The proposed framework could be possibly generalised to other road networks in similar topographical and geological regional areas. [ABSTRACT FROM AUTHOR]

The fourth industrial revolution creates ubiquitous sensor data in production plants. To generate maximum value out of these data, reliable and precise time series-based machine learning methods like temporal neural networks are needed. This paper proposes a novel sequence-to-sequence deep learning architecture for time series segmentation called PrecTime which tries to combine the concepts and advantages of sliding window and dense labeling approaches. The general-purpose architecture is evaluated on a real-world industry dataset containing the End-of-Line testing sensor data of hydraulic pumps. We are able to show that PrecTime outperforms five implemented state-of-the-art baseline networks based on multiple metrics. The achieved segmentation accuracy of around 96% shows that PrecTime can achieve results close to human intelligence in operational state segmentation within a testing cycle. [ABSTRACT FROM AUTHOR]

Malware detection approaches can be classified into two classes, including static analysis and dynamic analysis. Conventional approaches of the two classes have their respective advantages and disadvantages. For example, static analysis is faster but cannot detect the malware variants generated through code obfuscation, whereas dynamic analysis can effectively detect variants generated through code obfuscation but is slower and requires intensive resources. This paper proposes a novel deep learning-based approach for malware detection. It delivers better performance than conventional approaches by combining static and dynamic analysis advantages. First, it visualises a portable executable (PE) file as a coloured image. Second, it extracts deep features from the colour image using fine-tuned deep learning model. Third, it detects malware based on the deep features using support vector machines (SVM). The proposed method combines deep learning with machine learning and eliminates the need for intensive feature engineering tasks and domain knowledge. The proposed approach is scalable, cost-effective, and efficient. The detection effectiveness of the proposed method is validated through 12 machine learning models and 15 deep learning models. The generalisability of the proposed framework is validated on various benchmark datasets. The proposed approach outperformed with an accuracy of 99.06% on the Malimg dataset. The Wilcoxon signed-rank test is used to show the statistical significance of the proposed framework. The detailed experimental results demonstrate the superiority of the proposed method over the other state-of-the-art approaches, with an average increase in accuracy of 16.56%. Finally, to tackle the problems of imbalanced data and the shortage of publicly available datasets for malware detection, various data augmentation techniques are proposed, which lead to improved performance. It is evident from the results that the proposed framework can be useful to the defence industry, which will be helpful in devising more efficient malware detection solutions. [Display omitted] • A novel hybrid framework combines deep transfer learning and machine learning for malware detection. • An image-based PE dataset is generated by transforming malicious and benign Windows executables into RGB-coloured images. • The second step involves a novel combination of deep learning and machine learning models for malware detection. • An in-depth analysis using 15 deep learning and 12 machine learning models is presented for malware detection. • The framework is scalable, cost-effective, and efficient. It eliminates the need for domain experts for reverse engineering tasks. [ABSTRACT FROM AUTHOR]

In digital healthcare applications, anomaly detection is an important task to be taken into account. For instance, in ECG (Electrocardiogram) analysis, the aim is often to detect abnormal ECG signals that are considered outliers. For such tasks, it has been shown that deep learning models such as Autoencoders (AEs) and Variational Autoencoders (VAEs) can provide state-of-the-art performance. However, they suffer from certain limitations. For example, the trivial method of threshold selection does not perform well if we do not know the reconstruction loss distribution in advance. In addition, since healthcare applications rely on highly sensitive personal information, data privacy concerns can arise when data are collected and processed in a centralized machine-learning setting. Hence, in order to address these challenges, in this paper, we propose AnoFed, a novel framework for combining the transformer-based AE and VAE with the Support Vector Data Description (SVDD) in a federated setting. It can enhance privacy protection, improve the explainability of results and support adaptive anomaly detection. Using ECG anomaly detection as a typical application of the framework in healthcare, we conducted experiments to show that the proposed framework is not only effective (in terms of the detection performance) but also efficient (in terms of computational costs), compared with a number of state-of-the-art methods in the literature. AnoFed is very lightweight in terms of the number of parameters and computation, hence it can be used in applications with resource-constrained edge devices. [ABSTRACT FROM AUTHOR]

The increase of road traffic in large cities during the last years has produced that long and short-term traffic flow forecasting is a critical need for the authorities. The availability of good traffic flow prediction methods is a must to make informed decisions concerning (punctual) traffic congestions. Previous work has shown that the accuracy of these methods decreases if we consider urban traffic and long-term predictions. In this paper we present a hybrid model, combining a Convolutional Neural Network and a Bidirectional Long–Short-Term Memory network, and apply it to long-term traffic flow prediction in urban routes. This model combines the capability of CNN to extract hidden valuable features from the input model and the capability of BiLSTM to understand the temporal context. In order to assess the usefulness of our model, we considered four streets of the city of Madrid with different characteristics and compared the results of our proposed model with the ones obtained by eight widely used baseline models. The results show that our hybrid model outperforms the baseline models with respect to three metrics commonly used in regression: mean absolute error, root mean squared error and accuracy. [ABSTRACT FROM AUTHOR]

Traffic flow prediction remains an ongoing hot topic in the field of Intelligent Transportation System. The state-of-the-art traffic flow prediction models can effectively extract both spatial and temporal features of traffic flow data, but ignore the correlation and external interference between traffic nodes. To this end, this paper proposes a novel method based on Spatial–Temporal Complex Graph Convolution Network (ST-CGCN) for traffic flow prediction. Specifically, we first constructs the distance matrix, the data correlation matrix, and the comfort measurement matrix according to the geographical locations, the historical data record, and the external interference between traffic nodes. Then, these three matrices are fused into a complex correlation matrix by introducing self-learning dynamic weights to improve the joint modeling ability of spatial–temporal features and external factors. Next, a spatial feature extraction module and a temporal feature extraction module are designed to characterize dynamic spatial–temporal features. The spatial feature extraction module consists of a graph convolution operator with a proposed complex correlation matrix and a residual unit. The temporal feature extraction module consists of a 3D convolution operator and a Long Short-Term Memory (LSTM). Experiments constructed on five real-world datasets demonstrate that the new proposed ST-CGCN is more effective than several existing deep learning based traffic flow prediction models. The key source code and data are available at https://github.com/Bounger2/ST-CGCN. [ABSTRACT FROM AUTHOR]

The development of an automated rail line defect classification system is of great benefit, as railway tracks must be periodically monitored and inspected to guarantee the safety of rail transportation. In this paper, an effective multi-scale residual convolutional network (MSRConvNet) model is proposed to classify the different types of railway track defects. The skip connections with residual learning blocks are used to increase the effectiveness of the network. The multi-scale convolutions are connected with parallel and two skip connections in the structure to distribute detailed feature maps with each other. Therefore, different scale feature maps can be extracted. The data augmentation method is performed to ensure a balanced class distribution and to eliminate the negative effect of the imbalanced dataset. The proposed model is compared with both benchmark deep learning models and the different variations of the designed network. The results verify that the proposed model can reach superior classification fulfillment, and the MSRConvNet provides an overall accuracy of 99.83%, precision of 99.83%, sensitivity of 99.83%, specificity of 99.94%, F1-score of 99.83%, and Matthew's correlation coefficient of 99.78% for four defect classes. • Multi-scale residual network is designed for railway track defect classification. • Hierarchical features are extracted with different level kernels. • Data augmentation is used to increase network performance. • Obtained results are compared with variations of multi-scale structures. [ABSTRACT FROM AUTHOR]

Electric vehicles (EVs) are considered emerging popularity due to their cost-saving and eco-friendliness nature. The charging of EVs with elevated efficiency is a major issue for users as the hardware facilities, like charging stations are limited. Thus, the EV charging scheduling model is required to discover the charging schedule for each EV. An optimization-driven framework for EV charge scheduling in a Vehicular ad hoc network (VANET) topology is presented in this paper. First, the network simulation with the charge station (CS) and the EV locations are performed. The Deep Maxout network (DMN) is employed for predicting the power. To charge at the charging station, the vehicles are scheduled based on various parametric factors. The proposed Fractional Feedback Tree Algorithm (FFTA) is used to schedule charges in the EV network. The Feedback Artificial Tree (FAT) algorithm and fractional calculus (FC) are combined to create the proposed FFTA. Additionally, a new model of the fitness function is created taking into account variables like the average waiting time, the distance, the predicted power, and the quantity of EVs that are requested to be charged. With the shortest distance of 11.00 km, the highest power of 8.17 J, the shortest average waiting time of 0.47 s, and less number of EVs charged at 4.4, the proposed FFTA offered improved performance. [Display omitted] [ABSTRACT FROM AUTHOR]

The paper proposes a surrogate model for liquid moulding of structural composites. A methodology is presented to simulate the dual-phase Darcy's flow in a heterogeneous porous medium. The approach is an encoder–decoder that receives as input a matrix of permeabilities and produces two scalar fields that represent the pressure and front flow. This model is trained with synthetic data generated with a computer fluid dynamics simulator. In this context, the lack of robustness of models trained with the popular L 2 and L 1 losses is highlighted and several enhancements to these baseline approaches are introduced. First, the study provides a piece-wise power-logarithmic loss that improves training in the presence of the bimodal distribution of error residuals produced by the dual-phase flow predictions. A non-uniform sampling strategy for the selection of time training snapshots is also included, which contributes to improve the prediction accuracy. The estimation of the front flow field is further refined with a multi-task training strategy. The introduction of these improvements in the baseline models reduce the relative error of the pressure and front flow fields by more than 50%, performing these simulations in a record time of 50 ms. The surrogate model is further evaluated as a digital twin to predict – in a real experiment – the location and spatial extent of race-tracking channels and regions with dissimilar degrees of permeability. [ABSTRACT FROM AUTHOR]

Visual tracking methods based on correlation filter (CF) are to estimate the location and the scale of tracking object within video sequences. However, there are still several inadequate hypotheses within the CF framework. These inadequate hypotheses include the constant Gaussian label map, the predetermined object location and the insufficient utilization for features. To further address the problems caused by these hypotheses, this paper proposes a novel separable-programming based probabilistic-iteration and restriction-resolving correlation filter (PRCF). The main innovation points of this work are: 1) the separable-programming based tracking framework is adopted to achieve adaptive regulation incorporation and to simultaneously address location prediction and filter training; 2) the probabilistic-iteration scheme is suggested to update the Gaussian label for each frame to solve the problem of ad-hoc label map; 3) the adaptive feature fusion measure is introduced to abate the effects of insufficient utilization for numerous features; 4) the convergence behavior of PRCF is proved and discussed through theoretical analysis and worst-case convergence rate calculation. Experiments have also been conducted to test the efficiency of suggested PRCF on 7 benchmarks: OTB100, TC128, VOT2016, VOT2019, UAV123, NFS and LaSOT. The results have indicated that: 1) the PRCF obtains favorable performances compared with other 15 state-of-the-art (SOTA) CF-based trackers; 2) the PRCF reports comparable results against 4 deep learning based trackers; 3) the PRCF achieves a real-time speed of 52 frames-pre-second (FPS) on 7 benchmarks averagely. Thus, the PRCF is qualified for practical target tracking scenarios such as video surveillance and unmanned aerial vehicles (UAVs). [ABSTRACT FROM AUTHOR]

Internet of Vehicles (IoV) is a paradigm of ITS that incorporates automobiles, transportation, information sharing, and traffic infrastructure management with the aim to improve safety on roads. Federated learning enables the Internet of Vehicles (IoV) networks with minimal data communication overhead and privacy infringement. The vehicles participate in model training for improving the model accuracy at the global level and enhancing road safety to a great extent. In this paper, we apply federated learning to driver behavior analysis in which the vehicles in the network collaborate to train CNN-LSTM and CNN-Bi-LSTM deep learning models for driver behavior classification (safe, unsafe, or fatigue) without sharing raw data. The CNN-LSTM and CNN-Bi-LSTM models have been implemented and analyzed using real-time driver's behavior data, collected using inbuilt smartphone sensors or vehicle onboard devices. The proposed model has been evaluated using the performance parameters viz., accuracy, Area Under Curve (AUC), loss, precision, and recall. Both IID and non-IID types of datasets have been considered for analyzing the diversity of the proposed model. It is evident that implementation of federated learning architecture provided similar parametric outcomes on IID and non-IID datasets in comparison to contemporary deep learning implementations on a non-distributed dataset. The proposed work aims towards designing a reliable intelligent transportation system using federated learning. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel EEG classification approach based on the Spatio-spectral feature, aiming to design a motor movement/imagery classification model that extracts multi-domain features with promising performance. Firstly, the difference between motor imagery (MI) tasks and real execution tasks was analyzed by calculating complexity measures of different frequency bands. Then, different connection patterns between MI tasks and real execution tasks were investigated by constructing the PLV and PLI networks. The results of the PLV and PLI brain networks showed that real execution tasks' network connections intensity was stronger than MI tasks, which meant these two networks could be used to distinguish the EEG signal features of different tasks. Afterward, to fully explain the multi-domain features of EEG signals, we fused the Phase Locking Value (PLV) and Phase-Lag Index (PLI) matrices (spatial-domain) under the subsets of frequency bands (frequency-domain) into a 3-D feature, namely the Spatio-spectral feature. Finally, a 3-D convolutional neural network combined with a long short-term memory (3DCNN-LSTM) was utilized to decode the feature. The results showed that the average accuracy of 10 subjects, 20 subjects, 50 subjects, 80 subjects, and 103 subjects was 85.88%, 83.09%, 76.30%, 75.02%, and 74.54%. Taken together, the proposed method provided promising classification accuracies, superior multi-domain features extraction ability, simpler structure, and robustness to classify the different motor movement/imagery tasks. The results contribute to our understanding of applying the deep learning method to decode EEG multi-domain features in the brain-computer interface (BCI) systems (e.g., MI, emotion recognition, and epileptic seizure classification). • Analyzed the difference between MI tasks and corresponding real execution tasks by calculating complexity measures of different frequency bands. The results showed that the differences between different tasks were more significant in the alpha (α) band in the PLV network and the gamma (γ) band in the PLI network. • Investigated different connection patterns between MI tasks and real execution tasks by constructing the PLV and PLI networks after thresholding. The results showed that real execution tasks' network connections intensity was stronger than MI tasks, which meant these two networks could be used to distinguish EEG signal features of different tasks. • Fused the PLV and PLI matrices (spatial-domain) under the subsets of frequency bands (frequency-domain) into a 3-dimensional feature, namely the Spatio-spectral feature. • Utilized a 3-dimensional convolutional neural network combined with a long short-term memory (3DCNN-LSTM) to decode and classify the Spatio-spectral feature. The average classification accuracy of 10 subjects, 20 subjects, 50 subjects, 80 subjects, and 103 subjects was 85.88%, 83.09%, 76.30%, 75.02%, and 74.54%, respectively. [ABSTRACT FROM AUTHOR]

Timely and effective flight regime recognition is one of the key tasks for structural usage monitoring, it can provide early warning to the dangerous regime. However, existing methods for flight regime recognition either rely on expert knowledge or ignore the continuity and decision boundary of the regimes, which limits their performance for complex regimes and hinders their deployment in practice. In this paper, we provided a brand-new solution for this problem and designed a deep temporal segmentation neural network to realize intelligent regime segmentation. Meanwhile, we revealed the long-tailed distribution of flight regimes and proposed class-wise dynamic group rebalance loss to keep inter-class accuracy balanced. To evaluate the effectiveness of the proposed model, we collected and elaborately annotated plentiful actual flight sorties data, including 11 flight regimes. Test results demonstrated that the model can automatically separate different regimes in a continuous flight sortie without any pre-processing and post-processing while extracting the accurate regime boundary and achieving 95.98% recognition accuracy. • Flight regime recognition from the perspective of temporal segmentation. • DTS-Net realizes continuous regimes recognition and boundaries positioning. • Class-wise dynamic group rebalance loss deals with long-tailed distributed regimes. • Without additional prior knowledge and post-processing. [ABSTRACT FROM AUTHOR]

Currently, most of the existing point cloud registration methods use stacked edge convolution as feature extractor, ignoring the importance of deep semantic information, meanwhile, the use of attention mechanism tends to increase the computational cost of the model and limits the matching effect. This paper proposes a dynamic graph aggregation framework for point cloud registration by building a dynamic deep network, which can capture richer semantic information and shape properties of point clouds. First, a hybrid feature extractor, which fully fuses local graph information and global graph information, is designed to obtain more discriminative feature descriptors. Second, a local graph neighborhood scoring module to update local graph features and achieve feature enhancement is built by learning the difference of similar point neighborhood features. Finally, a dual-constrained matching module is designed, which is used to measure the similarity of point pairs from the two aspects of Euclidean distance and affinity between features. The proposed framework achieves excellent registration performance, as demonstrated by experimental results on challenging 3D point cloud benchmarks. Especially in partial point cloud registration, MAE(R) achieved excellent results of 0.2614 and 0.2619 under unseen shapes and unseen categories. [ABSTRACT FROM AUTHOR]

Accurate multi-node load forecasting is the key to the safe, reliable, and economical operation of the power system. However, the dynamic nature of load and the coupling nature of networks are difficult to extract, making consistent and accurate forecasting of node load rather difficult. In this regard, this paper proposes a soft sharing multi-task deep learning method for multi-node load forecasting in the power system. It has the following aspects: (1) Considering the coupling characteristics of the node network, a multi-modal feature module, based on the inception strategy and gated temporal convolutional network (GTCN), is firstly designed to explore the coupling features implied in the node load data. (2) A novel multi-objective neural network model is proposed to achieve simultaneous prediction of multi-node load by integrating the multi-modal feature module and gated recurrent unit (GRU). For sharing the learning information of sub-networks, this paper uses the soft sharing mechanism to capture load features, which can better optimize the prediction task for each node load simultaneously. Load data from the New Zealand distribution network and AEMO are used to compare the proposed model's performance in various scenarios using regression metrics such as mean absolute percentage error (MAPE), Weighted Mean Accuracy (WMA), root mean squared logarithmic error (RMSLE), and Diebold–Mariano (DM). The simulation results show that the proposed method can explore the spatial–temporal coupling characteristics in multi-node load data. Compared with existing state-of-the-art multi-node load prediction methods, our proposed method's MAPE decrease 17.04% and 3.92% in Non-aggregation and Aggregation situations. • A novel deep multi-task learning method is proposed for the multi-node forecasting problem. • An IGTCN module is designed to extract the coupling features from different node loads. • The multi-task deep neural network exhibits the accuracy of multi-node load forecasting. [ABSTRACT FROM AUTHOR]

To estimate the parameters of an induction motor in a data-based manner, this paper proposes a new offline method to estimate rotor resistance and excitation inductance based on the deep-Q-learning approach. In this method, parameter estimation can be facilitated without disturbing the model error or operating state. To achieve this goal, three important elements, namely observation, action, and reward, are appropriately designed. To improve the robustness and accelerate the convergence, a new concept, denoted as Q-sensitivity, is proposed and investigated in detail. The experimental results show that a high-Q-sensitivity design can allow the proposed method to obtain a fast and torque-maximized estimation. Results from the comparative studies confirm the accuracy and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

After 2007–2008 crisis, it is clear that corporate credit scoring is becoming a key role in credit risk management. In this paper, we investigate the performances of credit scoring models applied to CDS data sets. The classification performance of deep learning algorithm such as deep belief networks with Restricted Boltzmann Machines are evaluated and compared with some popular credit scoring models such as logistic regression, multi-layer perceptron and support vector machine. The performance is assessed using the classification accuracy and the area under the receiver operating characteristic curve. It is found that DBN yields the best performance. [ABSTRACT FROM AUTHOR]

Since the generative adversarial network (GAN) was proposed in 2014, it has rapidly become a hot topic in the field of deep learning. In recent years, there are many optimizations for GAN, which are divided into three kinds, including the optimization of loss functions, external structure of network and internal structure of network. Few people optimizes GAN from internal structure of network, and because of the process of game, slow training speed is one of the biggest problems of GAN. This paper introduces the ideology of broad learning algorithm (Chen and Liu, 2017) to put forward the multi-judge generative adversarial network method based on different random features to enhance the quality of generative result and increase the training speed of GAN (Goodfellow et al., 2014). This model provides a method which reduce the input information of each discriminator to optimize the training process of GAN. The experiments on cifar10 and anime face dataset find that our model obtains a better performance than the GMAN model and Base model. This paper finds a group of hyper-parameters to enhance the BroadGAN. [ABSTRACT FROM AUTHOR]

Deep Reinforcement Learning systems are now a hot topic in Machine Learning for their effectiveness in many complex tasks, but their application in safety-critical domains (e.g., robot control or self-autonomous driving) remains dangerous without mechanism to detect and prevent risk situations. In Deep RL, such risk is mostly in the form of adversarial attacks, which introduce small perturbations to sensor inputs with the aim of changing the network-based decisions and thus cause catastrophic situations. In the light of these dangers, a promising line of research is that of providing these Deep RL algorithms with suitable defenses, especially when deploying in real environments. This paper suggests that this line of research could be greatly improved by the concepts from the existing research field of Safe Reinforcement Learning, which has been postulated as a family of RL algorithms capable of providing defenses against many forms of risks. However, the connections between Safe RL and the design of defenses against adversarial attacks in Deep RL remain largely unexplored. This paper seeks to explore precisely some of these connections. In particular, this paper proposes to reuse some of the concepts from existing Safe RL algorithms to create a novel and effective instance-based defense for the deployment stage of Deep RL policies. The proposed algorithm uses a risk function based on how far a state is from the state space known by the agent, that allows identifying and preventing adversarial situations. The success of the proposed defense has been evaluated in 4 Atari games. [ABSTRACT FROM AUTHOR]

Speaker counting is an important research area in sound forensics. There are limited speaker counting papers in the literature, as it is challenging to collect datasets. This work aims to collect a new overlapping speech signal dataset for speaker counting and propose a novel feature engineering model. In this work, textural feature extraction is based on the iconic James Webb space telescope; hence, this pattern is named James Webb Pattern (JWPat). A new speaker counting speech dataset comprising 3,121 speeches divided into 32 classes (the class number corresponded to the number of speakers) was collected. A new framework that mimics the deep learning model has been proposed to classify the collected speech classes. The proposed feature engineering model is self-organized and uses various mother wavelet functions to generate features at both low and high levels. We have obtained the best classification accuracy of 86.74% using the symlet4 mother wavelet function. Using our proposed framework, eight classification results have been calculated with accuracy ranging from 75.94% to 86.74%. This range is over 10% accuracy, and it demonstrates the effect of the mother wavelet function on the classification performance. Moreover, the feature extraction capability of the mirror of the James Webb telescope has been demonstrated. Our proposed method yielded 86.74% accuracy on a large dataset and indicated the success of our proposed model. [ABSTRACT FROM AUTHOR]

Human body pose estimation represented by joint rotations is essential for driving the virtual characters. The present paper developed a novel end-to-end point-to-pose mesh fitting network (P2P-MeshNet) to directly estimate the body joint rotations. P2P-MeshNet provided a strong collaboration between the deep learning network, an inverse kinematics network for body pose estimation (IKNet-body), and the self-correcting network, an iterative error feedback network (IEF). The introduced P2P-MeshNet was then applied to the free mocap (FreeMocap) dataset covering OpenPose 3D joint locations reconstructed from multi-view OpenPose 2D joint locations. The generated joint rotations were tested using the mean per joint position error (MPJPE), as well as the percentage of correct keypoints (PCK) along with the area under the PCK curve (AUC) with a threshold range of 0–60 mm after Procrustes aligned. Based on the compared metrics, P2P-MeshNet with 11.31 mm and 99.7% in estimate error and success rate as well as an AUC of 80.9 demonstrated a more consistent tool for future human body pose estimation. The runtime performance of 100 frames per second implied its potential application prospects. • A novel end-to-end point-to-pose mesh fitting network (P2P-MeshNet) was proposed. • P2P-MeshNet was applied to estimate the body joint rotations directly. • The challenge of real-time estimation and driving the virtual characters was solved. • P2P-MeshNet provided a significant qualitative and quantitative improvement over the existing methods. [ABSTRACT FROM AUTHOR]

TV program recommendation is important for users in the face of a huge amount of information data. The existing TV program recommendation mainly relies on a collaborative filtering method to recommend through interactive data between users and programs. Although some methods utilize auxiliary information to enrich semantic features, most of them only use a single data type, which cannot capture a more diverse feature representation of the user and program. In this paper, we propose a neural TV program recommendation model with multi-source heterogeneous data, which makes full use of the multi-source heterogeneous auxiliary information. Specifically, we combine heterogeneous features derived from auxiliary information to learn a deep program representation in the program encoder module. To more accurately capture user preferences, we further utilize the personalized attention mechanism to determine the importance of different programs to the user representation based on the interaction between users and programs in the user encoder module. Extensive experiments on a real dataset of the Chinese capital show that our model can effectively improve the performance of TV program recommendations compared to the existing models. [ABSTRACT FROM AUTHOR]

Building occupancy information could aid energy preservation while simultaneously maintaining the end-user comfort level. Energy conservation becomes essential since energy resources are scarce and human dependency on appliances is only exponentially increasing. While intrusive sensors (i.e., cameras and microphones) can raise privacy concerns, this paper presents an innovative non-intrusive occupancy detection approach using environmental sensor data (e.g., temperature, humidity, carbon dioxide (CO 2), and light sensors). The proposed scheme transforms multivariate time-series data into images for better encoding and extracting relevant features. The utilized image transformation method is based on data normalization and matrix conversion. Precisely, by representing time-series in 2D space, an encoding kernel can move in two directions while it can move only in one direction when applied to a 1D signal. Moreover, machine learning (ML) and deep learning (DL) techniques are utilized to classify occupancy patterns. Several simulations are used to evaluate the approach; mainly, we investigated pre-trained and custom convolutional neural network (CNN) models. The latter attained an accuracy of 99.00%. Additionally, pixel data are extracted from the generated images and subjected to traditional ML methods. Throughout the numerous comparison settings, it was observed that the latter strategy provided the optimal balance of 99.42% accuracy performance and minimal training time across the occupancy datasets. • Exhaustive literature review on occupancy detection schemes based on sensor data. • Image transformation technique based on data normalization and matrix conversion. • Assessing a variety of pre-trained and custom CNN models to predict occupancy status. • Pixel extraction to gather values and integrate them with conventional ML algorithms. [ABSTRACT FROM AUTHOR]

Deep anomaly detection (DAD) is essential in optimizing building energy management. Nonetheless, most existing works concerning this field consider unsupervised learning and involve the analysis of sensor readings through a one-dimensional (1D) energy time series, which limits the options for detecting anomalies within the building's energy consumption. To the best of the authors' knowledge, this paper presents the first study that explores using two-dimensional (2D) image representations as features of a supervised deep transfer learning (DTL) approach. Specifically, using 2D image representations allows taking advantage of any underlying data within the feature set, providing more possibilities to encode data and detect pertinent features which may not be considered in standard 1D time-series. Furthermore, the effects of using CNN activations as machine learning (ML) model features are also investigated to combine the advantages of both techniques. Additionally, the concept of layer and hyperparameter variation for the CNN model is also studied, with the objective of reducing the overall time computation and resource requirements of the proposed system. Hence, this makes our approach a candidate for edge-based applications. As per the conducted experiments, the top methodology rests at the F1-scores of 93.63% and 99.89% under simulated and real-world energy datasets, respectively. This involves using grayscale 2D image representations that combine CNN activations extracted from AlexNet and GoogleNet pre-trained models as features to a linear support vector machine (SVM) classifier. Finally, the comparison analysis with the state-of-the-art has shown the superiority of the proposed method in various assessment criteria. • Deep anomaly detection of energy consumption using energy time-series images. • Converting 1D energy time series into 2D representations. • 2D energy representations as features of a neural network-based transfer learning. • Using neural network activations as machine learning model features. • Combining neural network activations extracted from AlexNet and GoogleNet. [ABSTRACT FROM AUTHOR]

Stroke, categorized under cardiovascular and circulatory diseases, is considered the second foremost cause of death worldwide, causing approximately 11% of deaths annually. Stroke diagnosis using a Computed Tomography (CT) scan is considered ideal for identifying whether the stroke is hemorrhagic or ischemic. However, most methods for stroke classification are based on a single slice-level prediction mechanism, meaning that the most imperative CT slice has to be manually selected by the radiologist from the original CT volume. This paper proposes an integration of Convolutional Neural Network (CNN), Vision Transformers (ViT), and AutoML to obtain slice-level predictions as well as patient-wise prediction results. While the CNN with inductive bias captures local features, the transformer captures long-range dependencies between sequences. This collaborative local-global feature extractor improves upon the slice-wise predictions of the CT volume. We propose stroke-specific feature extraction from each slice-wise prediction to obtain the patient-wise prediction using AutoML. While the slice-wise predictions helps the radiologist to verify close and corner cases, the patient-wise predictions makes the outcome clinically relevant and closer to real-world scenario. The proposed architecture has achieved an accuracy of 87% for single slice-level prediction and an accuracy of 92% for patient-wise prediction. For comparative analysis of slice-level predictions, standalone architectures of VGG-16, VGG-19, ResNet50, and ViT were considered. The proposed architecture has outperformed the standalone architectures by 9% in terms of accuracy. For patient-wise predictions, AutoML considers 13 different ML algorithms, of which 3 achieve an accuracy of more than 90%. The proposed architecture helps in reducing the manual effort by the radiologist to manually select the most imperative CT from the original CT volume and shows improvement over other standalone architectures for classification tasks. The proposed architecture can be generalized for volumetric scans aiding in the patient diagnosis of head and neck, lungs, diseases of hepatobiliary tract, genitourinary diseases, women's imaging including breast cancer and various musculoskeletal diseases. Code for proposed stroke-specific feature extraction with the pre-trained weights of the trained model is available at: https://github.com/rishiraj-cs/StrokeViT_With_AutoML. [Display omitted] • Most methods for stroke classification are based on slice-level prediction mechanism. • An integration of CNN and ViT is proposed for slice-level prediction. • For patient-wise predictions, stroke-specific feature extraction is proposed. • AutoML is trained on the proposed domain-specific features extracted. [ABSTRACT FROM AUTHOR]

Multi-object tracking (MOT) is a highly valued and challenging research topic in computer vision. To achieve more robust tracking performance, recently published MOT methods tend to use anchor-free object detectors, which have the advantage of dealing with the identity ambiguity problem encountered by anchor-based methods in learning appearance features. However, in practical applications, it is found that the detection accuracy of the anchor-free object detector based on classical convolutional neural networks in crowded scenes will be significantly reduced. In order to have better detection and tracking performance in crowded scenes, this paper proposes an anchor-free joint detection and embedding (JDE) MOT method based on Transformer architecture, called Swin-JDE. The proposed method includes a novel Patch-Expanding module, which can improve the spatial information of feature maps by up-sampling processing through neural network learning and Einops Notation-based rearrangement to enhance the detection and tracking performance of the MOT model. In terms of training method, we propose a two-step training method that trains the detection branch separately from the appearance branch to enhance the detection robustness of anchor-free predictors. Furthermore, during the training process, we also propose an examination method to remove occluded targets from the training dataset to improve the accuracy of the appearance embedding layer. In terms of data association, we propose a new post-processing method, which simultaneously considers the three factors of detection confidence, appearance embedding distance, intersection-over-union (IoU) distance to match each tracklet and the detection information to improve the tracking robustness of the MOT model. Experimental results show that the proposed method achieves 70.38% multiple object tracking accuracy (MOTA) and 69.53% identification F1-score (IDF1) results in the MOT20 benchmark dataset, and the identification switch (ID Switch) is reduced to 2026. Compared with FairMOT, the proposed method improves MOTA and IDF1 by 8.58% and 2.23%, respectively. [ABSTRACT FROM AUTHOR]

Identification of exfoliated graphene flakes and classification of the thickness are important in the nanomanufacturing of advanced materials and devices. This paper presents a deep learning method to automatically identify and classify exfoliated graphene flakes on Si/SiO 2 substrates from optical microscope images. The presented framework uses a hierarchical deep convolutional neural network that is capable of learning new images while preserving the knowledge from previous images. The deep learning model was trained and used to classify exfoliated graphene flakes into monolayer, bi-layer, tri-layer, four-to-six-layer, seven-to-ten-layer, and bulk categories. Compared with existing machine learning methods, the presented method showed high accuracy and efficiency as well as robustness to the background and resolution of images. The results indicated that the pixel-wise accuracy of the trained deep learning model was 99% in identifying and classifying exfoliated graphene flakes. This research will facilitate scaled-up manufacturing and characterization of graphene for advanced materials and devices. • A machine intelligence approach is developed to classify exfoliated graphene flakes. • A hierarchical deep convolutional neural network is applied for the classification task. • The neural network can learn from new images while preserving the learned knowledge. • The approach is robust to the background and resolution of microscopy images. • Advanced vision techniques are incorporated to handle highly imbalanced datasets. [ABSTRACT FROM AUTHOR]

Remaining useful life (RUL) estimation, a key component in predictive maintenance (PdM), aims to reduce maintenance cycles in the prognostic health of mechanical equipment(s). Research directions using deep-learning-based RUL estimation often suffer from limited availability of degradation signals resulting inaccurate predictions. Until now, state-of-the-art models trained on large sets of natural images to classify objects have not been re-used to improve regression-based RUL estimation accuracies of mechanical equipment. Actually, this is a rarely researched topic in PdM. Inspired by transfer learning, we showcase that the knowledge learned by popular pre-trained models can be transferred to improve industrial machinery-based-maintenance decision-making. Accordingly, this paper proposes a novel multi-model data-fusion-based deep transfer learning (MMF-DTL) framework for improved RUL estimation of rolling bearings through degradation images (DI) and pre-trained deep convolutional neural networks (CNNs). After procuring the degradation signals, we obtain DIs incorporating sufficient deterioration information using Markov Transition Fields. Next, these DIs are input into a DTL network comprising three pre-trained CNNs, i.e., DenseNet201, VGG16, and ResNet50, designed in a parallel fashion, where each constituent fine-tunes a different count of layers. Subsequently, features extracted from each component model are fused in a weighted manner and passed onto several fully connected layers. The effectiveness of the proposed framework is validated using the PHM Challenge 2012 bearing degradation dataset. Compared to several state-of-the-art works, our approach improves ∼ 12. 57 % on error rate and ∼ 26. 04 % on MAE, suggesting it is practically feasible to grasp transferable attributes from a general-purpose related dataset to another with minimal dataset size. [ABSTRACT FROM AUTHOR]

Recently, developing automated video surveillance systems (VSSs) has become crucial to ensure the security and safety of the population, especially during events involving large crowds, such as sporting events. While artificial intelligence (AI) smooths the path of computers to think like humans, machine learning (ML) and deep learning (DL) pave the way more, even by adding training and learning components. DL algorithms require data labeling and high-performance computers to effectively analyze and understand surveillance data recorded from fixed or mobile cameras installed in indoor or outdoor environments. However, they might not perform as expected, take much time in training, or not have enough input data to generalize well. To that end, deep transfer learning (DTL) and deep domain adaptation (DDA) have recently been proposed as promising solutions to alleviate these issues. Typically, they can (i) ease the training process, (ii) improve the generalizability of ML and DL models, and (iii) overcome data scarcity problems by transferring knowledge from one domain to another or from one task to another. Although the increasing number of articles proposed to develop DTL- and DDA-based VSSs, a thorough review that summarizes and criticizes the state-of-the-art is still missing. To that end, this paper introduces, to the best of the authors' knowledge, the first overview of existing DTL- and DDA-based video surveillance to (i) shed light on their benefits, (ii) discuss their challenges, and (iii) highlight their future perspectives. [ABSTRACT FROM AUTHOR]

Human activity recognition has become a pillar of today intelligent Human–Computer Interfaces as it typically provides more comfortable and ubiquitous interaction. This paper proposes a novel fuzzy-based deep learning-based algorithm to predict future sequences of activities from a given sequence of daily living activities of a subject wearing a lower limb exoskeleton. The engineering application concerns the challenging task of recognizing locomotion activities of the wearer in real-time, which is needed to ensure appropriate control of the robot during daily living activities. Indeed, real-time locomotion activity recognition is very challenging for controlling lower-limb exoskeletons. The model proposes a new adaptive kernel, based on the data features derived from the fuzzy rules on the input sequences to enrich the features of the activity sequences. Then, a CNN is applied to extract local subsequences from the whole sequences to identify local patterns in the convolution window. Finally, an attention-based GRU is incorporated into the model to extract meaningful parts of the time-series sequences. The results show high accuracy in the estimation of the transition between gait modes which is critical to ensure smooth control of the exoskeleton. The performance of the model is evaluated using the dynamic activity data gathered from different subjects. The proposed model outperforms the traditional models used in the literature. [ABSTRACT FROM AUTHOR]

Assembly is an important process of aerospace industry which involves significant number of fastening elements such as bolts, washers and nuts, etc. Presently, the identification of these elements is done manually by humans. Since human operators are prone to errors, any fault in the assembly line can have a major impact on the overall efficiency and safety of an aerospace vehicle. In this paper, we present a deep learning and image processing-based approach for identification of mechanical fixation/fastening elements used in the aerospace assembly line. We propose YOLO-v5 algorithm to classify the components based on their head and lateral shape. We also propose an image processing method to estimate the spatial dimensions of the assembly line components including thread pitch. Moreover, this study also features the development of an image acquisition platform with two cameras mounted and proper lighting mechanism to capture quality images. Despite the challenges associated with such systems, the proposed deep learning (YOLO-v5) algorithm has shown promising results with a mAP@0.5 of 0.996 for component classification. Whereas, the proposed image processing mechanism achieved an accuracy of 100% for standard size assignments with a maximum error of 0.05 mm for pitch calculation. [ABSTRACT FROM AUTHOR]

As a significant extension of rough set theory, three-way decision (3WD) theory plays a crucial role in the data mining of uncertain information and decision-making analysis. Transfer learning (TL) is also a powerful knowledge discovery and deep learning strategy that has attracted the attention of many scholars. At present, many achievements have been made in related studies based on the concepts of sample transfer, feature transfer, and parameter transfer. However, there are few studies on multi-granularity fusion and TL for multisource data from the perspective of the stochastic dominance (SD) relation. In this paper, an intuitionistic fuzzy three-way transfer learning (IF3WTL) model based on rough almost stochastic dominance (RASD) is proposed. In this scenario, we first introduce the concept of a rough marginal information measure and the corresponding calculation method. Then, an RASD method is proposed to generate the multi-granularity distribution of same-category information in the source domain. In addition, 3WD theory is introduced to classify the target domain objects into positive, negative, and boundary regions based on the relation between marginal minimum risk information and the corresponding multi-granularity distribution. Furthermore, a secondary decision strategy with an SVM algorithm is implemented to iteratively process boundary region objects. The proposed method can reduce the differences in the data distribution among domains through RASD, improve noise tolerance, and obtain the degree of roughness of common information at different scales. Moreover, the proposed method adopts an iterative learning strategy, which can reduce the decision-making cost in cases with insufficient information and improve the accuracy of classification for objects in the boundary region. The rationality and effectiveness of the proposed model are verified through experiments with the ABIDE dataset and comparative analyses with the existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

While constructing predictive model for industrial process quality prediction, the selection of an appropriate input variable set is crucial to the online prediction performance. In this paper, the issue is solved through a deep causality analysis method, where the deep learning feature extracting unit, i.e., the Gated Recurrent Unit (GRU), is combined with the Granger Causality (GC) to extract the causal relationships between process variables. In this way, the nonlinear and dynamic features contained in the process data can be effectively learned in the GC analyzing framework, resulting in a more reliable causal relationship structure. Furthermore, through applying the attention mechanism to the input layer of the GRU cells, the causal relationship between process variables and the quality variable can be extracted, even the long time-lags exist. Based on the direct and indirect causal structure, the quality-related causal input variables are selected, and the predictive model is established through a deep dynamic regression network with GRU cells. A numerical example and a real chemical process case are provided to verify the effectiveness of the proposed causal variable selection method, where the prediction performance is significantly improved comparing with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Community detection is a field of research with increasing influence due to its importance in dimensionality reduction and revealing the organizational patterns in networks. A community or cluster in a network reflects the interactions among the objects that are strongly connected and distinguish them from the ones in other clusters. Over the past decades, various algorithms have been developed to detect the community structure in networks. However, the performance of these algorithm is affected by the existence of noise or nonlinear information in the network. Consequently, deep learning algorithms arise to tackle this issue. In this paper, we propose a novel deep robust auto-encoder nonnegative matrix factorization (DRANMF) approach to detect the community structure in networks. In particular, DRANMF consists of a deep structured decoder and encoder components to transform the high-order proximity matrix of the network to and from the cluster membership space, respectively. Moreover, we enhance the robustness of the proposed approach against noise and outliers by adopting the l 2 , 1 -norm to formulate the objective function. The experiments on multiple synthetic and real-world networks demonstrate that the proposed DRANMF has a better performance and robustness than other existing methods. [ABSTRACT FROM AUTHOR]

In the pathological diagnosis of early, late and non-membranous nephropathy, direct immunofluorescence is highly likely to present potentially specific lesions, while it is often overlooked due to the difficulty of screening with naked eyes. With the advanced progress of deep learning, they have shown powerful abilities in detecting potential lesions. In this paper, we propose an adaptive weighted confidence calibration fusion framework (Ada-CCFNet) consisting of a preprocessing module, an adaptive weighted confidence calibration fusion (Ada-CCF) module and a classification module for diagnosis of membranous nephropathy by classifying the multimodal direct immunofluorescence images. In the preprocessing module, we use the well-known U-Net to segment individual glomeruli and standardize their luminance appearance by the average luminance difference method, allowing the subsequent modules to focus more on the diseased glomerular region. Subsequently, in the Ada-CCF module, six confidence calibration methods are utilized for two main direct immunofluorescence images, IgG and C3, and the comprehensive calibration scores are obtained based on the adaptive weighted fusion of six confidence calibration methods to obtain more reliable confidence level, in which the adaptive weights are related with expected calibration error reductions. For the classification module, the weighted probability scores of IgG and C3 are jointly fed into the module to achieve the classification by random forest. Experimental results showed that Ada-CCFNet achieves the classification accuracy of 73.52%, surpassing the methods of using single IgG or C3 images and positive grade indicator with 8.24%, 8.94% and 22.76%, and outperforming the compared methods in the classification of membranous nephropathy. • An adaptive weighted confidence calibrated fusion network (Ada-CCFNet) is designed to classify early, late and non-membranous nephropathy. • Classify membranous nephropathy stages by multi-modal direct immunofluorescence images. • A preprocessing method to segment glomerular region effectively and standardize luminance. • An adaptive weighted method by Excepted Calibration Error reductions is proposed for fusion. • Our method provides higher classification accuracy than the compared methods. [ABSTRACT FROM AUTHOR]

Deep learning models have gained a reputation of high accuracy in many domains. Convolutional Neural Networks (CNN) are specialized towards image recognition and have high accuracy in classifying objects within images. However, CNNs are an example of a black box model, meaning that experts are unsure how they work internally to reach a classification decision. Without knowing the reasoning behind a decision, there is low confidence that CNNs will continue to make accurate decisions, so it is unsafe to use them in high-risk or safety–critical​ fields without first developing methods to explain their decisions. This paper is a survey and analysis of the available explainability methods for showing the reasoning behind CNN decisions. [ABSTRACT FROM AUTHOR]

Modern power systems are experiencing the challenge of high uncertainty with the increasing penetration of renewable energy resources and the electrification of heating systems. In this paradigm shift, understanding electricity users' demand is of utmost value to the retailers, aggregators, and policymakers. However, behind-the-meter (BTM) equipment and appliances at the household level are unknown to the other stakeholders mainly due to privacy concerns and tight regulations. In this paper, we seek to identify residential consumers based on their BTM equipment, mainly rooftop photovoltaic (PV) systems and electric heating, using imported/purchased energy data from utility meters. To solve this problem with an interpretable, fast, secure, and maintainable solution, we propose an integrated method called Interpretable Refined Motifs And binary Classification (IRMAC). The proposed method comprises a novel shape-based pattern extraction technique, called Refined Motif (RM) discovery, and a single-neuron classifier. The first part extracts a sub-pattern from the long time series considering the frequency of occurrences, average dissimilarity, and time dynamics while emphasising specific times with annotated distances. The second part identifies users' types with linear complexity while preserving the transparency of the algorithms. With the real data from Australia and Denmark, the proposed method is tested and verified in identifying PV owners and electrical heating system users. The performance of the IRMAC is studied and compared with various state-of-the-art methods. The proposed method reached an accuracy of 96% in identifying rooftop PV users and 94.4% in identifying electric heating users, which is comparable to the best solution based on deep learning, while the speed of the inference model is a thousand times faster. Last but not least, the proposed method is a transparent algorithm, which can tackle the concerns regarding the agnostic decision-making process when policies prohibit some machine learning methods. [ABSTRACT FROM AUTHOR]

The high-speed railway is a deeply coupled system with strong and weak electrical equipment, while complex electromagnetic interference (EMI) consequently brings potential risks and hazards to signaling safety. Since the incident reports on signaling failure intrinsically reflect the generation and evolution mechanism of equipment failures, relying on text mining technology, this paper tries to extract failure-related entities and constructs a knowledge graph to clarify the negative impact of the on-site electromagnetic environment. Firstly, based on convolutional neural networks (CNN), a supervised deep learning model for Chinese text classification is established to generate a corpus containing only railway failures caused by EMI. Then, the bidirectional long short-term memory (BiLSTM) and bidirectional encoder representations from transformers (BERT) algorithms are adopted to build the named entity recognition (NER) model. A NER algorithm more suitable for Chinese text features is proposed through ensemble modeling, training verification, and comparative evaluation. Finally, the knowledge storage and visualization of relational graph construction based on the Neo4j database are realized according to the obtained failure-related entities. This knowledge topology network effectively explores the inherent relationship between EMI factors and railway safety, as well as provides support for improving the safety assessment and enhancing the anti-interference performance of the equipment. • Electromagnetic interference as a hidden risk may lead to railway safety incidents. • Text mining-based approach for understanding railway incident reports is proposed. • BERT is applied to improve the performance of NER for Chinese text. • A knowledge graph of fault-related entities is implemented based on Neo4j. [ABSTRACT FROM AUTHOR]