Search

In this paper, we show that adversarial training time attacks by a few pixel modifications can cause undesirable overfitting in neural networks for both discriminative and generative models. We propose an evolutionary algorithm to search for an optimal pixel attack using a novel cost function inspired by domain adaptation literature to design our training time attack. The proposed cost function explicitly maximizes the generalization gap and domain divergence between clean and corrupted images. Empirical evaluations demonstrate that our adversarial training attack can achieve significantly low testing accuracy (with high training accuracy) on multiple datasets by just perturbing a single pixel in the training images. Even under the use of popular regularization techniques, we identify a significant performance drop compared to clean data training. Our attack is more successful than previous pixel-based training time attacks on state-of-the-art Convolutional Neural Networks (CNNs) architectures, as evidenced by significantly lower testing accuracy. Interestingly, we find that the choice of optimization plays an essential role in robustness against our attack. We empirically observe that Stochastic Gradient Descent (SGD) is resilient to the proposed adversarial training attack, different from adaptive optimization techniques such as the popular Adam optimizer. We identify that such vulnerabilities are caused due to over-reliance on the cross-entropy (CE) loss on highly predictive features. Therefore, we propose a robust loss function that maximizes the mutual information between latent features and input images, in addition to optimizing the CE loss. Finally, we show that the discriminator in Generative Adversarial Networks (GANs) can also be attacked by our proposed training time attack resulting in poor generative performance. Our paper is one of the first works to design attacks for generative models.

Increasing human activities have caused serious disturbance to global forest resources, so how to accurately identify individual trees has become an important task of forest resources investigation. In order to get the accurate number of individual trees, this paper took coniferous forest and mixed coniferous and broad-leaved forest as experimental samples, as well as Digital Orthophoto Map and airborne LiDAR Point Cloud as research data. We propose a deep Learning individual tree segmentation method based on RetinaNet model and PCS algorithm by doing comparative analysis (classical Watershed Algorithm and Layer Stacking Algorithm) at the plots (with high, medium, and low densities). The experimental results show that the method proposed in this paper can solve the problem of individual tree segmentation in high density forest and improve its degree of automation. Compared with Watershed algorithm and Layer stacking algorithm, F-Measure is improved by 6%-29% and 7%-20%, respectively. In other words, the results of individual tree segmentation presented in this paper can not only improve the precision of individual tree segmentation, but also maintain a high detection rate, which can meet the accuracy and high efficiency of individual tree extraction, so as to meet the needs for modern forestry investigation.

With the rapid development of remote-sensing imaging technology, remote-sensing images have become increasingly diverse, and people are paying more attention to ocean remote-sensing research. Because ocean remote-sensing data are complex, and the ocean environment is diverse, results will differ, even if the same target is detected at different times in the same scene. To obtain more semantic features and better pixel-level prediction capabilities, this paper proposes a pixel-level ocean remote-sensing image algorithm (GLPO-Net) that combines local and global features. First, texture features, color features, and spatial relationship features are extracted. Second, the algorithm constructs a multiscale local cross-attention mechanism strategy to obtain feature weight information in different directions to fully mine the local features of ocean remote-sensing images. Concurrently, an algorithm constructs a multiscale global cross-attention mechanism strategy to obtain global features. Then, the fusion of global features and local features is described in each submodule to obtain more representative deep features. Finally, small-sample ocean remote-sensing is described via image pixel-level prediction. The algorithm proposed in this paper has been tested with three public ocean remote-sensing datasets. The experimental results show that the proposed GLPO-Net algorithm can learn features from small samples of ocean remote-sensing images. Compared to the prediction results of other remote-sensing image algorithms, GLPO-Net exhibits better prediction capabilities.

Accurate diagnosis of corn crop diseases is a complex challenge faced by farmers during the growth and production stages of corn. In order to address this problem, this paper proposes a method based on K-means clustering and an improved deep learning model for accurately diagnosing three common diseases of corn leaves: gray spot, leaf spot, and rust. First, to diagnose three diseases, use the K-means algorithm to cluster sample images and then feed them into the improved deep learning model. This paper investigates the impact of various k values (2, 4, 8, 16, 32, and 64) and models (VGG-16, ResNet18, Inception v3, VGG-19, and the improved deep learning model) on corn disease diagnosis. The experiment results indicate that the method has the most significant identification effect on 32-means samples, and the diagnostic recall of leaf spot, rust, and gray spot disease is 89.24 %, 100 %, and 90.95 %, respectively. Similarly, VGG-16 and ResNet18 also achieve the best diagnostic results on 32-means samples, and their average diagnostic accuracy is 84.42% and 83.75%. In addition, Inception v3 (83.05%) and VGG-19 (82.63%) perform best on the 64-means samples. For the three corn diseases, the approach cited in this paper has an average diagnostic accuracy of 93%. It has a more significant diagnostic effect than the other four approaches and can be applied to the agricultural field to protect crops.

Dementia diseases are increasing rapidly, according to the World Health Organization (WHO), becoming an alarming problem for the health sector. The electroencephalogram (EEG) is a non-invasive test that records brain electrical activity and has a wide field of applications in the medical area, one of which is the detection of neurodegenerative diseases. The aim of this work is to present the results of a thorough review of the use of EEG systems for the detection of dementia diseases. Around 82 papers published between 2009 and 2020 were reviewed and compared obtaining data such as sampling time, number of electrodes, the most popular processing, classification, and validation techniques, as well as an analysis of the reported results. The relationship of the selected parameters with the efficiency obtained is shown. Some more common combinations in the reviewed papers that demonstrated to have reliability levels greater than 90%, and details to be considered at each stage of the process. An overview of the most commonly used classification tools and processing techniques is also described.

Autonomous driving (AD) provides a reliable solution for safe driving by replacing human drivers responsible for the majority of accidents. The emergence of Machine Learning, specifically Deep Reinforcement Learning (DRL), and its ability to solve complex games proved its potential to address AD challenges. However, model-free methods still suffer from safety-related issues that can be resolved using safe-DRL approaches. The addition of model-based safety filters to the learning-based algorithms provides safety bounds on their performance and constraint satisfaction. In this paper, we investigate the addition of a safety filter based on Model Predictive Control and show an increase in mean testing episode reward by 110% from −75 mean episode reward during testing for 50 episodes for Deep Deterministic Policy Gradient ( $DDPG$ ) to 7.758. We study the impacts of safety filters (7.758 mean reward), heuristic rules, bounded additive noises (0.49% performance increase comparing to noise-free case), and exploration (3.425 mean reward) on the learning algorithm. We compare the effects of filters in the context of simulated exploration and bounded exploration and prove that bounded exploration results in 9.86% increase in mean reward and 12.95% decrease in std comparing to the other method. Additionally, inspired by Deep Internal Learning and biological mechanisms like brain plasticity, we investigate the idea of using each sample for training only once instead of utilizing stochastic batches which increases the mean testing accumulated reward by 1.87% and leads to the best performance (7.942 mean reward and 0.048 std). Finally, the results demonstrate better automotive results for our proposed method than DDPG. Our proposed method, DDPG with safety filter in bounded exploration and adaptive learning under noisy input conditions, has a success rate of 100% under different traffic densities for the simulation environment used in this paper and our assumptions. The proposed method’s automotive results are shown for a braking scenario to avoid collision with other road users.

Choosing optimal Deep Learning (DL) architecture and hyperparameters for a particular problem is still not a trivial task among researchers. The most common approach relies on popular architectures proven to work on specific problem domains led on the same experiment environment and setup. However, this limits the opportunity to choose or invent novel DL networks that could lead to better results. This paper proposes a novel approach for providing general recommendations of an appropriate DL architecture and its hyperparameters based on different configurations presented in thousands of published research papers that examine various problem domains. This architecture can further serve as a starting point of investigating DL architecture for a concrete data set. Natural language processing (NLP) methods are used to create structured data from unstructured scientific papers upon which intelligent models are learned to propose optimal DL architecture, layer type, and activation functions. The advantage of the proposed methodology is multifold. The first is the ability to eventually use the knowledge and experience from thousands of DL papers published through the years. The second is the contribution to the forthcoming novel researches by aiding the process of choosing optimal DL setup based on the particular problem to be analyzed. The third advantage is the scalability and flexibility of the model, meaning that it can be easily retrained as new papers are published in the future, and therefore to be constantly improved.

Accurate electricity consumption forecasting in the power grids ensures efficient generation and distribution of electricity. Keeping this in mind, the paper introduces a novel deep learning model, termed Gated-FCN, for short-term load forecasting. The key idea is to introduce an automated feature selection and deep learning model for forecasting. The model includes an eight-layered Fully Convolutional Network (FCN-8) in which the hand-crafted feature selection that requires expert domain knowledge is avoided. Furthermore, the model also reduces noise as it learns internal dependencies and the correlation of the time series. Enhanced Bidirectional Gated Recurrent Unit (EBiGRU) is used in combination with FCN-8 to learn long-term temporal dependencies of the time series. Moreover, a weighted averaging mechanism of multiple snapshot models is adopted in the proposed model to assign optimized weights to BiGRU. At the end of FCN-8 and BiGRU, a fully connected dense layer is used that gives final prediction results. Gated-FCN is an end-to-end forecasting model that does not require any other model for enhancing its forecasting efficiency. Different activation functions are initially analyzed to determine how the proposed model learns complex patterns from the time series data. Later, the activation function having the best accuracy is used for forecasting. The proposed model extracts both spatial and temporal features from the data. Furthermore, this paper also provides predictive and exploratory data analyses to assist policymakers in making optimal decisions regarding power production and dispatch. In order to demonstrate the applicability of the proposed technique, the simulations are performed using nine years’ load consumption data taken from Independent System Operators New England (ISO-NE). The comparison with five state-of-the-art techniques is also provided to prove the fact that Gated-FCN gives the best forecasting accuracy as compared to other benchmark techniques in terms of two performance metrics: Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE).

Many studies showed the feasibility of detecting Freezing of Gait (FOG) of Parkinson’s patients by using several numbers of inertial sensors worn on the body and in back-end computing power. This work uses machine learning approaches for analyzing the data of one single body-worn inertial sensor system to classify and detect FOG. Long-Short-Term-Memory (LSTM) is employed as the FOG detection algorithm and the Daphnet (FOG and normal gait) dataset provides the data for model training and testing in this paper. The model considers raw data from three channels of the acceleration sensor mounted on the patient’s shank and ignores all other data from other sensors. The model is patient dependent and uses sensitivity and specificity metrics to evaluate the model’s performance. In this paper, we propose a novel padding method that is applied to the windows of FOG and non-FOG with zero overlaps on the training set and adapts the padding to the individual regions. This method produces windows of only one type of data and label. The proposed padding method reduces the padding amount by two orders of magnitude compared to bigger batch sizes in the sequence splitting method offered by MATLAB 2019a. The padding amount is independent of the batch size. Raw data is fed to the model in the testing mode without any pre-processing or data transformation. The standard rolling window generates fixed-size windows for the test set without overlap and the higher amount of FOG or Normal walking data which defines the label of the individual window. The model for one-second long windows applied in this work outperformed the literature results with a sensitivity of 92.57% and a specificity of 95.62% compared to 82% and 94% reported by Masiala et al.

Laparoscopic surgery is a surgical procedure performed by inserting narrow tubes into the abdomen without making large incisions in the skin. It is done with the aid of a video camera. Laparoscopic videos are affected by various distortions during surgery which lead to loss of visual quality. Identification of these distortions is the primary requisite in automated video enhancement systems used to classify the distortions correctly and accordingly select the proper algorithm to enhance video quality. In addition to high accuracy, the speed of distortion classification should be high, and the system must consider real-time conditions. This paper aims to address the issues faced by similar methods by developing a fast and accurate deep learning model for distortion classification. The dataset proposed by the ICIP2020 conference challenge was used for training and evaluation of the proposed method. This challenging dataset contains videos that have five types of distortions such as noise, smoke, uneven illumination, defocus blur, and motion blur with four levels of intensity. This paper discusses the proposed solution which received the first prize in the ICIP2020 challenge. The solution utilized a transfer learning approach to transfer representation from the domain of natural images to the domain of laparoscopic videos. We used a pre-trained ResNet50 convolutional neural network (CNN) to extract informative features that were mapped by support vector machine (SVM) classifiers to various distortion categories. In this work, the problem of multiple distortions in the same video was formulated as a multi-label distortion classification problem. The approach of transfer learning with decision fusion was applied and was found to outperform other solutions in terms of accuracy (83%), F1 score of a single distortion (94.7%), and F1 score of single and multiple distortions (94.9%). In addition, the proposed solution can run in real time with an inference speed of 20 frames per second (FPS).

In this paper, we propose a highly efficient method for synthesizing high-resolution(HR) smoke simulations based on deep learning. A major issue for physics-based HR fluid simulations is that they require large amounts of physical memory and long execution times. In recent years, this issue has been addressed by developing deep-learning-based super-resolution(SR) methods that convert low-resolution(LR) fluid simulation results to HR(High-resolution) versions. However, these methods were not very efficient because they performed operations even in areas with low density or no density. In this paper, we propose a method that can maximize its efficiency by introducing a downscaled and binarized adaptive octree. However, even if it is divided by octree, because the number of nodes increases when the resolution of the simulation space is large, we reduce the size of the space by multiscaling and at the same time perform binarization to preserve the density that may be lost in this process. The octree calculated in this process has a structure similar to that of a multigrid solver, and the octree calculated at coarse resolution is restored to its original size and used for HR expression. Finally, we apply the SR process only to those areas having significant density values. Using the proposed method, the SR process is significantly faster and the memory efficiency is improved. The performance of our method is compared with that of an existing SR method to demonstrate its efficiency.

Generally, a few-shot distribution shift will lead to a poor generalization. Furthermore, while the number of instances of each class in the real world may significantly different, the existing few-shot classification methods are based on the assumption that the number of samples in each class is equal, which causes the trained classifier invalid. Moreover, through ResNet and WRN (Wide Residual Network) have achieved great success in the image processing field, the depth and width of CNNs constrain the conventional convolution layer performance. Thus, to overcome the above problems, the model of this paper proposes a novel few-shot classification model that uses learning balance variables to decide how much to learn from the imbalance dataset, which dynamically generates the convolution kernel based on each input. In our model, to extend the decision boundaries and enhance the class representations, this paper uses embedding propagation as a regularizer for manifold smoothing. Manifold smoothing can effectively solve the above problems of transductive learning. The interpolations between neural network features based on similarity graphs are used by embedding propagation. Experiments show that embedding propagation can produce a better embedding manifold and our model in standard few-shot datasets, such as miniImagenet, tieredImagenet, CUB has state-of-the-art results. It significantly outperforms the existing few-shot approaches, which consistently improves the accuracy of the models by about 11%.

This paper proposed a novel fuzzy best-worst multi-criteria group decision-making method to solve the group decision-making (GDM) problem with multi-granular linguistic approach, which is an effective and promising technique to tackle this issue. In the proposed method, the selectable multi-granularity linguistic term sets (LTS) are firstly provided for experts to expressed their individual assessment information. Then, the improved fuzzy BWM is employed to calculate the weights of criteria with the form of fuzzy numbers. In current several studies using the BWM for group decision-making, only two unified best and worst criteria are given, which cannot reflect the evaluation of the best and worst criteria by different experts, resulting in the omission of information. Moreover, the difference between the best and worst criteria initially given and the experts’ ideas will cause the experts to be inaccurate in the comparison of each criterion. Therefore, in this paper, in order not to omit too much information, each expert will determine the best and the worst criteria. The evaluation information of each expert is integrated into two comparison vectors according to the transformation formula proposed in this paper. What’s more, an improved input-based consistency measurement is proposed, which can provide the DMs with a clear guideline on the revision of the inconsistent judgement(s). Finally, two examples are given to illustrate the effectiveness and applicability of the proposed method.

This article mainly studies Electroencephalogram (EEG) mental recognition. Because the human brain is very complex and the EEG signal is greatly affected by the environment, EEG mental recognition can be attributed to domain adaptative problems. Our main work is as follows: (1) At present, most domain adaptation learning only learns the linear subspace of Reproducing Kernel Hilbert Space (RKHS), and RKHS itself does not. Given the complexity and nonlinearity of EEG mental recognition, we propose an EEG mental recognition algorithm based on two learning. The two learning is RKHS learning and RKHS subspace learning. The source dictionary regularized RKHS subspace learning we proposed applies to EEG mental recognition and is better than pure RKHS subspace learning, but not enough. To get satisfactory results, we learn RKHS before RKHS subspace. (2) According to Moore–Aronszajn theorem, RKHS can be uniquely generated by a kernel function. The existing RKHS is rarely learnable. It is difficult to find a kernel function that can be learned and optimized. In RKHS learning, this paper uses a learnable kernel function that we published, and the kernel function is easy to optimize. (3) In RKHS subspace learning, most of the existing methods adopt the Maximum Mean Discrepancy (MMD) criterion, but it cannot make the spatial distribution of the same category of source and target domain data overlap as much as possible, and the label of the target domain data is unknown. To solve this problem, this paper proposes a framework of RKHS subspace learning based on source domain dictionary regularization. The experimental results on the brain-computer interface international competition data set (BCI competition IV 2a) show that the effect of the algorithm proposed in this paper is better than that of the other five advanced domain adaptation learning algorithms.

DNA microarray data analysis is infamous due to a massive number of features, imbalanced class distribution, and limited available samples. In this paper, we focus on high-dimensional multi-class imbalanced problems. The high dimensional and multi-class imbalanced problem has posed acute challenges for the conventional classifiers to effectively perform classification tasks on both the minority and majority classes. Numerous efforts have been devoted to addressing either high dimensionality dataset or class imbalance problems. Nonetheless, few methods have been proposed to address the intersection of multi-class imbalanced and high-dimensional problems concurrently due to their intricate interactions. This paper presents novel hybrid algorithms for feature selection with the high dimensional multi-class imbalanced problem using multiple filter-based rankers (MFR) and hybrid Grasshopper optimization algorithm (GOA). The Simulated Annealing (SA) algorithm is incorporated into GOA. SA is used to enhance the best solution found by the GOA algorithm. The aim of using the SA here is to tackle the slow convergence and improve the exploitation by searching the high-quality regions found by the GOA. The experimental results confirm the effectiveness of the proposed methods in improving the classification performance in terms of area under the curve (AUC) compared to other well-known methods, which guarantees the ability of the proposed methods in searching the feature space and identifying very robust and discriminative features that best predict the minority class.

In recent times, numerous convolutional neural network (CNN) based detection models have been proposed and have shown excellent performance. However, because these models are generally developed to detect objects in class units (e.g., person, car), additional training processes with numerous datasets are required to find a specific object. This paper proposes a model that accurately detects specific persons by using top clothing color information without any additional training processes. The proposed method combines CNN-based instance segmentation and pose estimation, utilizing all the advantages of each technique. To avoid redundant computations, these two schemes are implemented as a filtering-based sequential operation structure. As a result, the proposed method has a 92.57% of accuracy in detecting a specific person with only a slight processing speed decrease. Furthermore, in this paper, the proposed model is efficiently ported on the heterogeneous embedded platform (i.e., NVIDIA Jetson AGX Xavier) with a parallel processing technique to maximize the hardware utilization.

In a smart urban environment, providing accurate information about human activities is an important task. It is a trend to implement human activity recognition (HAR) algorithms and applications on smart phones, including health monitoring, self-management system, health tracking and so on. However, human activity recognition(HAR) is very complex, and it is important to use the best technology and machine learning to understand human activities. One of the main problems of the existing HAR strategies is that the classification accuracy is relatively low, and in order to improve the accuracy, it needs high computational overhead. The purpose of this paper is to use an exploratory data analysis method to deal with HAR, after the implementation of different data mining techniques, the results of dimensionality reduction and visualization are obtained. The HAR method based on smart phone and EDA proposed in this paper is a high precision method. Compared with other classifiers, its accuracy is 96.56%. This article will discuss computational prediction activities and the computational limitations of using exploratory data to analyze (EDA) on 564 feature data frames. The experimental results show that HAR-based exploratory data analysis is a common sensory signal processing technology. The GridSearchCV and LinearSVC algorithm can provide accurate automatic human activity recognition (HAR), for elderly and disabled patients in need of continuous care, and it is a decision-making tool to support sports coaches to make plans.

Artificial intelligence-assisted diagnosis systems are developing rapidly, but doctors are currently less aware of artificial intelligence-assisted diagnosis systems. Understanding how to allow doctors to accept and use artificial intelligence medical assistant diagnosis system can promote the implementation of artificial intelligence medical assistant diagnosis system applications. Taking into account the current difficulties faced by the Internet of Things medical consultation services, this paper proposes a business operation model based on multi-party participation and sharing of medical consultation resources. We designed the information flow, overall logic and service implementation process of the service model, and completed the construction of the artificial intelligence medical service service model. We combine IoT technology to build a vital signs monitoring environment and clarify how to use IoT devices. In the error backpropagation algorithm, there is no significant difference in the contribution of different samples to the weight change, which makes the adjustment of network parameters not easily affected by difficult medical consultation samples, thereby weakening the effect of network medical consultation. In order to solve this problem, this article defines the degree to which the sample belongs to its correct category as the confidence of medical inquiry, and divides the training sample into a dangerous sample and a safe sample according to a dynamic threshold. Based on the difficulty of medical inquiry, an improved new learning algorithm is proposed. The algorithm penalizes the loss of dangerous samples, so that the convolutional neural network pays more attention to dangerous samples and can learn more effective information. Aiming at the eight physiological characteristics of data, this paper adjusts the structure of the convolutional neural network to make it take into account the richness of data characteristics and the dynamics of data changes over time. The realized CNN optimization algorithm model has improved the prediction effect, and the accuracy rate of medical consultation reaches 90.15%, which is better than other machine learning algorithms.

With the advent of smart grid, load forecasting is emerging as an essential technology to implement optimal planning and control of grid assets. Ergo in recent years, a significant thrust can be witnessed for the research towards the improvement of the prediction of the energy demand. However thus far there has not been any one technique in the literature that is shown to give best forecasts for a variety of sites; almost all the papers published on load forecasting, report their best results on just one of the dataset. This problem accentuates further when the training data does not have enough data points to learn patterns over all the seasons. Hence to devise a load forecasting technique that can yield the best estimates on diverse datasets, especially when the training data is limited, is a big challenge, which is addressed in this paper. The paper presents a novel combination of deep learning with feature engineering for short-term load forecasting. The proposed architecture, named as Deep Derived Feature Fusion (DeepDeFF), is based on the sequential model in conjunction with the hand-crafted derived features in order to aid the model for better learning and predictions. The raw data and the hand-crafted features are trained at separate levels, then their respective outputs are combined to make the final prediction. The efficacy and robustness of the proposed methodology is evaluated on diverse datasets from five countries with completely different patterns. The extensive experiments and results demonstrate that the proposed technique is superior to the existing state of the art.

This paper presents a hybrid ontology-XLNet sentiment analysis classification approach for sentence-level aspects. The main objective of the proposed approach allows discovering user social data considering the extracted in-depth inference about sentiment depending on the context. Thus, in this paper, we investigate the contribution of utilizing the lexicalized ontology to improve the aspect-based sentiment analysis performance through extracting the indirect relationships in user social data. The XLNet model is utilized for extracting the neighboring contextual meaning and concatenating it with each embeddings word to produce a more comprehensive context and enhance feature extraction. In the proposed approach, Bidirectional Long Short Term Memory (Bi-LSTM) networks are used for classifying the aspects in online user reviews. Various experiments considering Adverse Drug Reactions (ADRs) discovery are conducted on six drug-related social data real-world datasets to evaluate the performance of the proposed approach using several measures. Obtained experimental results show that the proposed approach outperformed other tested state-of-the-art related approaches through improving feature extraction of unstructured social media text and accordingly improving the overall accuracy of sentiment classification. A significant accuracy of 98% and F-measure of 96.4% are achieved by the proposed ADRs aspect-based sentiment analysis approach.

Prescription dispensing accuracy is of paramount importance for all hospitals. However, human errors are inevitable due to multiple reasons, such as fatigue, stress, heavy workload, lack of effective verification measures, mismanagement. Such human errors pose serious safety and health concerns on the part of patients and may as well lead to a series of medical disputes. Based on induced deep learning, this paper proposes a real-time Blister Package Identification System (BPIS) to assist pharmacists’ drug verification and dispensing. Under the guidance of the induction strategy, image preprocessing is introduced to form a standardized image containing the front and back side of the blister package, which is subsequently sent to CNN-based object identification network for feature extraction and identification. This preprocessing method allows the identification system to promote the deep learning system to focus on feature learning to obtain more information about the appearance of the package ruling out confounding factors such as background noise, size, shape or positioning. In addition, this article collects and establishes an image dataset of adult lozenges. Under this dataset, this paper verifies the enhancement of Induced Deep Learning (IDL) on YOLO v2, ResNet, and SENet. By optimizing the deep learning identification network with the help of the embedded technology and a two-side extraction mechanism, a real-time BPIS is built. Long-term tests in hospitals prove the effectiveness of the proposed system.

Document layout analysis plays an important role in the area of Document Understanding. It is responsible for identifying and classifying the different components of digital documents. Currently, there is no universal algorithm that fits all types of digital documents. This work presents a novel approach for identifying tables, figures, isolated equations and text regions in scientific papers using deep learning and computer vision techniques. Our proposed approach is a three-stage system: (i) obtaining the spectrograms of the horizontal and vertical intensity histograms of segmented regions of interest; (ii) labeling segmented regions of interest into text, table, and figures using a deep convolutional neural network classifier; and (iii) identifying isolated equations in text regions using Bag of Visual Words (BOVW) with Zernike moments. We built a new dataset composed of 11007 papers to perform the experiments, using two common segmentation metrics to evaluate our model: (1) Adjusted Rand Index (ARI) and (2) Variation of Information (VI). The proposed document layout analysis system reached an overall accuracy of 96.2685%, outperforming prior art with a less computational cost.

Recent advances in the domain of software defect prediction (SDP) include the integration of multiple classification techniques to create an ensemble or hybrid approach. This technique was introduced to improve the prediction performance by overcoming the limitations of any single classification technique. This research provides a systematic literature review on the use of the ensemble learning approach for software defect prediction. The review is conducted after critically analyzing research papers published since 2012 in four well-known online libraries: ACM, IEEE, Springer Link, and Science Direct. In this study, five research questions covering the different aspects of research progress on the use of ensemble learning for software defect prediction are addressed. To extract the answers to identified questions, 46 most relevant papers are shortlisted after a thorough systematic research process. This study will provide compact information regarding the latest trends and advances in ensemble learning for software defect prediction and provide a baseline for future innovations and further reviews. Through our study, we discovered that frequently employed ensemble methods by researchers are the random forest, boosting, and bagging. Less frequently employed methods include stacking, voting and Extra Trees. Researchers proposed many promising frameworks, such as EMKCA, SMOTE-Ensemble, MKEL, SDAEsTSE, TLEL, and LRCR, using ensemble learning methods. The AUC, accuracy, F-measure, Recall, Precision, and MCC were mostly utilized to measure the prediction performance of models. WEKA was widely adopted as a platform for machine learning. Many researchers showed through empirical analysis that features selection, and data sampling was necessary pre-processing steps that improve the performance of ensemble classifiers.

Large-scale group decision-making (LSGDM) is one of the main open problems where a decision is made by many different results. Moreover, there is also a problem with how to make the decision when there is no all information. This uncertainty can be very problematic for many different solutions in artificial intelligence. In this paper, we propose to extend a federated learning (FL) approach to not only a training process but also for making a decision using many different classifiers. This solution is applied in LSGDM, where many different results are intended for the classification of various data and can be used for deciding, even when some of the data are missing. For this purpose, we propose a fuzzy consensus that can be used in these problems. The contribution of this paper is the new way of using FL and extending its operation to many different classifiers. Our proposition was described for medical purposes and evaluated to show the advantages of the proposal. The proposal obtained 89,12% of accuracy on HAM10000, which is one of the best results compared to state-of-art.

Predicting the personality type of text authors has a well-known usage in psychology with practical applications in business. From the data science perspective, we can look at this problem as a text classification task that can be tackled using natural language processing (NLP) and deep learning. This paper proposes a method and a novel loss function for multiclass classification using the Myers–Briggs Type Indicator (MBTI) approach for predicting the author’s personality type. Furthermore, this paper proposes an approach that improves the current results of the MBTI multiclass classification because it considers components of compound class labels as supportive elements for better classification according to MBTI. As such, it also provides a new perspective on this classification problem. The experimental results on long short-term memory (LSTM) and convolutional neural network (CNN) models outperform baseline models for multiclass classification, related research on multiclass classification, and most research with four binary approaches to MBTI classification. Moreover, other classification problems that target compound class labels and label parts with binary mutually exclusive values can benefit from this approach.

In this paper, we propose a classification system that uses multiple autoencoder models for identifying malware images. It is crucial to accurately classify malware before we can deploy appropriate countermeasures to prevent them from spreading. Rapid malware classification is the first step in preparing effective countermeasures. Typical approaches to this problem, which can be divided into static or dynamic methods, are not suitable for efficient malware classification because they require either fixed malware patterns or lots of time to investigate, respectively. If the malware analysts have enough time and resources, they can analyze any malware thoroughly. However, finite resources mean they always suffer from a lack of time due to the malware that needs analyzing increasing at a dramatic rate. In the real world, new malware and variants of existing malware are constantly emerging. To address this issue, many researchers have developed approaches using machine learning techniques. However, to date these systems have had difficulty responding appropriately to the rapidly changing malware environment and also suffer from data imbalance problems in the training data. The system proposed in this paper consists of multiple autoencoder models that classify malware that has been converted to an image. Each autoencoder model classifies only one type of malware and is trained using only samples from the corresponding family, this allows the system to update quickly and mitigates the data imbalance problem. We demonstrate our method’s superior performance through various experiments compared to other state-of-the-art techniques using the Malimg dataset.

he acoustic emission technique has been applied successfully for the identification, characterization, and localization of deformations in civil engineering structures. Numerous localization techniques, such as Modal Acoustic Emission, Neural Networks, Beamforming, and Triangulation methods with or without prior knowledge of wave velocity, have been presented. Several authors have conducted in-depth research in the localization of AE sources. However, existing review papers do not focus on the performance evaluation of existing source localization techniques. This paper discusses these techniques based on the number of sensors used and the geometry of the structures of interest. Furthermore, it evaluates them on the basis of their performance. At the end of this paper, a comparative analysis of existing methods has been presented based on their basic principles, key strengths, and limitations. A deep learning circular sensor cluster-based solution has the potential to provide a low-cost reliable localization solution for acoustic emission sources.

This paper addresses the problems related to the mapless navigation control of wheeled mobile robots based on deep learning technology. The traditional navigation control framework is based on a global map of the environment, and its navigation performance depends on the quality of the global map. In this paper, we proposes a mapless Light Detection and Ranging (LiDAR) navigation control method for wheeled mobile robots based on deep imitation learning. The proposed method is a data-driven control method that directly uses LiDAR sensors and relative target position for mobile robot navigation control. A deep convolutional neural network (CNN) model is proposed to predict motion control commands of the mobile robot without the requirement of the global map to achieve navigation control of the mobile robot in unknown environments. While collecting the training dataset, we manipulated the mobile robot to avoid obstacles through manual control and recorded the raw data of the LiDAR sensor, the relative target position, and the corresponding motion control commands. Next, we applied a data augmentation method on the recorded samples to increase the number of training samples in the dataset. In the network model design, the proposed CNN model consists of a LiDAR CNN module to extract LiDAR features and a motion prediction module to predict the motion behavior of the robot. In the model training phase, the proposed CNN model learns the mapping between the input sensor data and the desired motion behavior through end-to-end imitation learning. Experimental results show that the proposed mapless LiDAR navigation control method can safely navigate the mobile robot in four unseen environments with an average success rate of 75%. Therefore, the proposed mapless LiDAR navigation control system is effective for robot navigation control in an unknown environment without the global map.

Image super-resolution reconstruction has been widely used in remote sensing, medicine and other fields. In recent years, due to the rise of deep learning research and the successful application of convolutional neural networks in the image field, the super-resolution reconstruction technology based on deep learning has also achieved great development. However, there are still some problems that need to be solved. For example, the current mainstream image super-resolution algorithms based on single or multiple frames pursue high performance indicators such as PSNR and SSIM, while the reconstructed image is relatively smooth and lacks many high-frequency details. It is not conducive to application in a real environment. To address such problem, this paper proposes a super-resolution reconstruction model of sequential images based on Generative Adversarial Networks (GAN). The proposed approach combines the registration module to fuse adjacent frames, effectively use the detailed information in multiple consecutive frames, and enhances the spatio-temporality of low-resolution images in sequential images. While the GAN was used to improve the effect of image high-frequency texture detail reconstruction, WGAN was introduced to optimize model training. The reconstruction results not only improved the PSNR and SSIM indexes but also reconstructed more high-frequency detail textures. Finally, in order to further improve the perception effect, an additional registration loss item RLT is introduced in the GAN network perception loss. Through extensive experiments, it shows that the model proposed in this paper effectively obtains the information between the sequence images. When the PSNR and SSIM indicators are improve, it can reconstruct better high-frequency texture details than the current advanced multi-frame algorithms.

This paper proposes a human gait estimation algorithm using a multiple 360 degree 2D LiDARs system. The system is fixed on the ground at the shin height to scan human legs during gait. The multiple LiDARs system is to overcome the drawbacks of a single LiDAR system, which could lose data due to occlusion between legs during walking and has a short tracking range. The performance of a sensor fusion system strongly depends on the calibration. In this paper, we propose a calibration method using a cylinder with known radius as a specific marker. In contrast to other methods, the calibration parameters and the cylinder center points at different positions are estimated by a proposed iterative algorithm. The measurement noises in the LiDAR output are considered to increase the accuracy of calibration and human leg center points estimation. Instead of using least square fitting of circle algorithm to estimate the leg center point, a new iterative algorithm which includes measurement noises is proposed. Although multiple LiDARs are used, the discontinuities of leg center points could still happen. Therefore, a quadratic optimization based eighth-order splines algorithm is derived to interpolate and smooth the data. Two configurations of three LiDARs are tested in the experiment. The former is the triangle configuration in which the whole walking path is covered by all three LiDARs. This configuration minimizes the occlusion between legs. The maximum RMSE of step length estimation of this configuration compared with the optical camera system is 0.03m. The latter is the line configuration in which each LiDAR covers a certain walking path sequentially. This configuration maximizes the tracking range. The experiment with 20m straight walking has the RMSE of about 0.10m.

A novel method for detection of important scenes in baseball videos based on correlation maximization between heterogeneous modalities via bidirectional time lag aware deep multiset canonical correlation analysis (BiTl-dMCCA) is presented in this paper. The proposed method enables detection of important scenes by collaboratively using baseball videos and their corresponding tweets. The technical contributions of this paper are twofold. First, since there are time lags between not only “tweets and corresponding multiple previous events” but also “events and corresponding multiple following posted tweets”, the proposed method considers these bidirectional time lags. Specifically, the representation of such bidirectional time lags into the derivation of their covariance matrices is newly introduced. Second, the proposed method adopts textual, visual and audio features calculated from tweets and videos as multi-modal time series features. Important scenes are detected as abnormal scenes via anomaly detection based on a generative adversarial network using multi-modal features projected by BiTl-dMCCA. The proposed method does not need any training data with annotation. Experimental results obtained by applying the proposed method to actual baseball matches show the effectiveness of the proposed method.

Recently, there is a rapid and wide increase in fake news, defined as provably incorrect information spread with the goal of fraud. The spread of this type of misinformation is a severe danger to social cohesiveness and well-being since it increases political polarisation and people’s distrust of their leaders. Thus, fake news is a phenomenon that is having a significant impact on our social lives, particularly in politics. This paper proposes novel approaches based on Machine Learning (ML) and Deep Learning (DL) for the fake news detection system to address this phenomenon. The main aim of this paper is to find the optimal model that obtains high accuracy performance. Therefore, we propose an optimized Convolutional Neural Network model to detect fake news (OPCNN-FAKE). We compare the performance of the OPCNN-FAKE with Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and The six regular ML techniques: Decision Tree (DT), logistic Regression (LR), K Nearest Neighbor (KNN), Random Forest (RF), Support Vector Machine (SVM), and Naive Bayes (NB) using four fake news benchmark datasets. Grid search and hyperopt optimization techniques have been used to optimize the parameters of ML and DL, respectively. In addition, N-gram and Term Frequency—Inverse Document Frequency (TF-IDF) have been used to extract features from the benchmark datasets for regular ML, while Glove word embedding has been used to represent features as a feature matrix for DL models. To evaluate the performance of the OPCNN-FAKE, accuracy, precision, recall, F1-measure were applied to validate the results. The results show that OPCNN-FAKE model has achieved the best performance for each dataset compared with other models. Furthermore, the OPCNN-FAKE has a higher performance of cross-validation results and testing results over the other models, which indicates that the OPCNN-FAKE for fake news detection is significantly better than the other models.

Colorization is a process of converting grayscale images into visually acceptable color images. The main goal is to convince the viewer of the authenticity of the result. Grayscale images that need to be colorized are, in most cases, images with natural scenes. Over the last 20 years a wide range of colorization methods has been developed – from algorithmically simple, yet time- and energy-consuming because of unavoidable human intervention to more complicated, but simultaneously more automated methods. Automatic conversion has become a challenging area that combines machine learning and deep learning with art. This paper presents an overview and evaluation of grayscale image colorization methods and techniques applied to natural images. The paper provides a classification of existing colorization methods, explains the principles on which they are based, and highlights their advantages and disadvantages. Special attention is paid to deep learning methods. Relevant methods are compared in terms of image quality and processing time. Different metrics for color image quality assessment are used. Measuring the perceived quality of a color image is challenging due to the complexity of the human visual system. Multiple metrics used to evaluate colorization methods provide results by determining the difference between the predicted color value and the ground truth, which in several cases is not in coherence with image plausibility. The results show that user-guided neural networks are the most promising category for colorization because they successfully combine human intervention and neural network automation.

Identifying k ey frames is the first and necessary step before solving the variety of other B haratanatyam problems. The paper aims to partition the momentarily stationary frames (key frame s) from this dance video’s motion frames. The proposed key frame s (KFs) localization is novel, simple, and effective compared to the existing dance video analysis methods. It is distinctive from standard KFs detection algorithms as used in other human motion videos. In the dance’s basic structure, the occurrence of KFs during performances is often not completely stationary and varies with the dance form and the performer. Hence, it is not easy to decide a global threshold (on the quantum of motion) to work across dancers and performances. The earlier approaches try to compute the threshold iteratively. However, the novelty of the paper is: (a) formulating an adaptive threshold, (b) adopting Machine Learning (ML) approach and, (c) generating the effective feature by combining three frame differencing and bit-plane technique for the KF detection. In ML, we use Support Vector Machine (SVM) and Convolutional Neural Network (CNN) as the classifiers. The proposed approaches are also compared and analyzed with the earlier approaches. Finally, the proposed ML techniques emerge as a winner with around 90% accuracy.

Partial discharge (PD) detection and diagnosis based on the ultra-high frequency (UHF) signals is one of the most widely adopted methods to evaluate the internal insulation status of high voltage equipment. Benefit from the rapid development of computing hardware and data processing algorithms, the intelligent PD fault diagnosis method based on the UHF data has made considerable progress in the past two decades. This two-part paper aims to give a comprehensive review about the application of signal processing and machine learning technologies in UHF PD detection and diagnosis. These technologies are divided into three categories according to their respective purpose, which are the preprocessing technology, source localization technology and pattern recognition technology. As the first one of the two-part review, we focus on the preprocessing and localization approaches in this paper. Specifically, for the preprocessing topic, the methods for signal denoising, multi-source separation, and pulse segmentation are included. While for the localization topic, the time difference of arrival (TDOA) method, direction of arrival (DOA) method, received signal strength indicator (RSSI) method, and other latest methods are reviewed. For each topic, the basic ideas, recent research progresses, advantages and limitations are discussed in detail. Before the conclusion, we also make a discussion about the application effects of the above technologies and prospect some future directions accordingly. In the second paper, the pattern recognition problems based on the UHF PD data will be concentrated, especially the application of deep learning algorithms.

Speaker recognition is related to human biometrics dealing with the identification of speakers from their speech. Speaker recognition is an active research area and being widely investigated using artificially intelligent mechanisms. Though speaker recognition systems were previously constructed using handcrafted statistical means of machine learning, currently it is being shifted to state-of-the-art deep learning strategies. Further, deep learning being a fast-paced domain, an absence of comprehensive survey is observed in the current deep speaker recognition technologies. In this paper, we focus on deep speaker recognition technologies. The paper particularly introduces a taxonomy, explains the progress, architectural strategies and processes of some distinctive approaches. Further, the manuscript classifies and enlists the currently available datasets and programming tools. Finally, the paper investigates the challenges and future directives of deep speaker recognition technology.

Despite the growing interest in the interplay of machine learning and optimization, existing contributions remain scattered across the research board, and a comprehensive overview on such reciprocity still lacks at this stage. In this context, this paper visits one particular direction of interplay between learning-driven solutions and optimization, and further explicates the subject matter with a clear background and summarized theory. For instance, machine learning and its offsprings are trending because of their enhanced capabilities in automating analytical modeling. In this realm, learning-based techniques (supervised, unsupervised, and reinforcement) have grown to complement many of the optimization problems in testing and training. This paper overviews how machine learning-based techniques, namely deep neural networks, echo-state networks, reinforcement learning, and federated learning, can be used to solve complex and analytically intractable optimization problems, for which specific cases are examined in this paper. The paper particularly overviews when learning-based algorithms are useful at solving particular optimizing problems, especially those of random, dynamic, and mathematically complex nature. The paper then illustrates such applications by presenting particular use-cases in communications and signal processing including wireless scheduling, wireless offloading and resource management, power control, aerial base station placement, virtual reality, and vehicular networks. Lastly, the paper sheds light on some future research directions, where the dynamicity and randomness of the underlying optimization problems make deep learning-driven techniques a necessity, namely in sensing at the terahertz (THz) bands, cellular vehicle-to-everything, 6G communication networks, underwater optical networks, distributed optimization, and applications of emerging learning-based techniques.

Due to the variety of video types and different quality on the Internet, it brings more challenges to video processing algorithms such as video object segmentation. Most existing video object segmentation methods rely on modules in other fields as an additional structure of the segmentation model. The combination of modules can improve the accuracy of the model, but it will also reduce the algorithm speed. This paper proposes a semi-supervised video object segmentation method based on local and global consistency learning, which does not rely on additional structures to achieve fast segmentation. First, we extract the embedding features of the image based on GhostNet which is the lightweight network. By using the embedded features of pixels, the graph model is established based on the similarity between pixels. Second, we adopt the local-global consistency learning framework to construct the label conduction model. Third, to optimize the memory occupation and inference speed of the model, we propose a sampling strategy for reference frames by considering local and global information. Finally, we establish a high-speed monitoring video dataset to verify the practical application effect of the method. Our method achieves a result of 69.5% $J\& F$ mean with 46 FPS on DAVIS 2017 dataset. At the same time, this paper constructed a high-speed monitoring video dataset. The algorithm obtained 68.2% $J\& F$ on this dataset, indicating that the method has good generalization and robust performance in practical applications.

Violence against women and children is a public health issue of pandemic proportions. It is estimated that one in every three women worldwide has experienced physical, emotional, or sexual violence. Similarly, each year one out of two children are victims of some form of violence including domestic aggression and bullying. Due to the widespread use of the Internet and social media, women and children are now vulnerable to other types of violence such as cyber-bullying and online sexual or emotional harassment. To help alleviate this social problem, the use of computer science and related technologies has been leveraged in recent years. The Internet of Things, artificial intelligence, ubiquitous and mobile computing, pattern recognition, cloud computing and similar technologies, have been used to formulate solutions to detect and prevent violent acts against women and children. In this paper, a systematic review of some of the efforts that can help address the problem of violence against women and children is presented. This paper describes the current state-of-the-art of these contributions and identifies trends, architectures, technologies, and current open challenges. The survey was developed using a literature review of academic documents published from 2010 to 2020. The contributions were categorized into four application domains: online detection, offline detection, safety, and education. These contributions were further categorized based on the computer science approaches and technologies used: artificial intelligence, Internet of Things, and digital serious games.

Predefined evenly-distributed class centroids (PEDCC) can be widely used in models and algorithms of pattern classification, such as CNN classifiers, classification autoencoders, clustering, and semi-supervised learning, etc. Its basic idea is to predefine the class centers, which are evenly-distributed on the unit hypersphere in feature space, to maximize the inter-class distance. The previous method of generating PEDCC uses an iterative algorithm based on a charge model, that is, the initial values of various centers (charge positions) are randomly set from the normal distribution, and the charge positions are updated iteratively with the help of the repulsive force between charges of the same polarity. The class centers generated by the algorithm will produce some errors with the theoretically evenly-distributed points, and the generation time will be longer. This paper takes advantage of regular polyhedron in high-dimensional space and the evenly distribution of points on the n dimensional hypersphere to generate PEDCC mathematically. Then, we discussed the basic and extensive characteristics of the frames formed by PEDCC. Finally, experiments show that new algorithm is not only faster than the iterative method, but also more accurate in position. The mathematical analysis and experimental results of this paper can provide a theoretical tool for using PEDCC to solve the key problems in the field of pattern recognition, such as interpretable supervised/unsupervised learning, incremental learning, uncertainty analysis and so on.

The rate of development of robotic technologies has been meteoric, as a result of compounded advancements in hardware and software. Amongst these robotic technologies are active exoskeletons and orthoses, used in the assistive and rehabilitative fields. Artificial intelligence techniques are increasingly being utilised in gait analysis and prediction. This review paper systematically explores the current use of intelligent algorithms in gait analysis for robotic control, specifically the control of active lower limb exoskeletons and orthoses. Two databases, IEEE and Scopus, were screened for papers published between 1989 to May 2020. 41 papers met the eligibility criteria and were included in this review. 66.7% of the identified studies used classification models for the classification of gait phases and locomotion modes. Meanwhile, 33.3% implemented regression models for the estimation/prediction of kinematic parameters such as joint angles and trajectories, and kinetic parameters such as moments and torques. Deep learning algorithms have been deployed in ~15% of the machine learning implementations. Other methodological parameters were reviewed, such as the sensor selection and the sample sizes used for training the models.

SAR images contain a large amount of noise, and related algorithms will cause high complexity when increasing the accuracy. To overcome this problem, a neural network model based on the attention mechanism was proposed in this paper. The model extracted information in two stages. It gradually extracts high-level features by reducing noise first and then adding hybrid attention. First, use dual-channel one-dimensional convolution to reconstruct the residual shrinkage network to construct a lightweight and efficient feature module, which improved the information extraction of the module with the consumption of a small amount of computing resources. Then, it was used as the backbone for model construction. Subsequently, mixed adaptive pooling was adopted to improve the maximum pooling. After that, dimensionality was reduced by pooling and linear interpolation was used to increase dimensionality, so as to generate feature weights of mixed dimension. Tests were performed on MSTAR dataset. The results showed that compared with the advanced algorithms, the proposed model in this paper can greatly reduce the amount of parameters and complexity while ensuring accuracy. The robustness test demonstrated that the model can effectively identify images with noise being added.

The Canny algorithm is widely used for edge detection. It requires the adjustment of parameters to obtain a high-quality edge image. Several methods can select them automatically, but they cannot cover the diverse variations on an image. In the Canny algorithm, we need to set values of three parameters. One is related to smoothing window size, and the other two are the low and high threshold. In this paper, we assume that the smoothing window size is fixed to a predefined size. This paper proposes a method to provide adaptive thresholds for the Canny algorithm, which operates well on images acquired under various variations. We select optimal values of two thresholds adaptively using an algorithm based on the Deep Q-Network (DQN). We introduce a state model, a policy model, and a reward model to formulate the given problem in deep reinforcement learning. The proposed method has the advantage that it can adapt to a new environment using only images without labels, unlike the existing supervised way. We show the feasibility of the proposed algorithm by diverse experimental results.

An end-to-end process to achieve a complete framework methodology for Harmful Algal Bloom (HAB) growth prediction is crucial for water management, especially in implementing robust predictive modelling of HAB to prevent water pollution. Previous works have separately focused on the prediction part or the implementation of the water monitoring system that involves the integration of sensors through the Internet of Things (IoT). These studies lack in terms of discussion of both IoT with the algae ecological domain and prediction method. Therefore, this paper takes the initiative to provide a wider coverage on the end-to-end process including the assembly and integration of sensors, data acquisition and predictive modelling using data-driven approaches, for example, machine learning, deep learning and deep time series forecasting algorithm for future algal bloom outbreak mitigation. This paper believes that discussion in a complete framework perspective based on the execution of each phase is important besides providing a true understanding of the algae growth factors and prediction problems to achieve a robust prediction algorithm for algal growth. In the end, this paper presents proof that selecting the right features and utilising time series with deep learning are much better for tackling the issues of highly non-linear and dynamic algae ecological data that are briefly introduced in this paper. Among all the algorithms selected, Long Short-term Memory (LSTM) is the best fit for the prediction method and has outperformed other basic machine learning methods in accurately predicting algal growth through the prediction of chlorophyll-a (Chl-a) as a strong indicator of algal presence for coastal studies.

A number of recent papers have shown experimental evidence that suggests it is possible to build highly accurate deep neural network models to detect COVID-19 from chest X-ray images. In this paper, we show that good generalization to unseen sources has not been achieved. Experiments with richer data sets than have previously been used show models have high accuracy on seen sources, but poor accuracy on unseen sources. The reason for the disparity is that the convolutional neural network model, which learns features, can focus on differences in X-ray machines or in positioning within the machines, for example. Any feature that a person would clearly rule out is called a confounding feature. Some of the models were trained on COVID-19 image data taken from publications, which may be different than raw images. Some data sets were of pediatric cases with pneumonia where COVID-19 chest X-rays are almost exclusively from adults, so lung size becomes a spurious feature that can be exploited. In this work, we have eliminated many confounding features by working with as close to raw data as possible. Still, deep learned models may leverage source specific confounders to differentiate COVID-19 from pneumonia preventing generalizing to new data sources (i.e. external sites). Our models have achieved an AUC of 1.00 on seen data sources but in the worst case only scored an AUC of 0.38 on unseen ones. This indicates that such models need further assessment/development before they can be broadly clinically deployed. An example of fine-tuning to improve performance at a new site is given.

The most important parts of rotating machinery are the rolling bearings. Finding bearing faults in time can avoid affecting the operation of the entire equipment. The data-driven fault diagnosis technology of bearings has recently become a research hotspot, and the starting point of research is often the acquisition of vibration signals. There are many public data sets for rolling bearings. Among them, the most widely used public dataset is Case Western Reserve University bearing center (CWRU). This paper will start from the CWRU data set, compare and analyze some basic methods of machine learning based rolling bearing fault diagnosis, and summarize the characteristics of CWRU. First, we give a comprehensive introduction to CWRU and summarize the results achieved. After that, the basic methods and principles of machine learning based rolling bearing fault diagnosis were summarized. Finally, we conduct experiments and analyze experimental results. This paper will have certain guiding significance for the future use of CWRU for machine learning based rolling bearing fault diagnosis.

The selection of a meta-learner determines the success of a stacked ensemble as the meta-learner is responsible for the final predictions of the stacked ensemble. Unfortunately, in imbalanced classification, selecting an appropriate and well-performing meta-learner of stacked ensemble is not straightforward as different meta-learners are advocated by different researchers. To investigate and identify a well-performing type of meta-learner in stacked ensemble for imbalanced classification, an experiment consisting of 19 meta-learners was conducted, detailed in this paper. Among the 19 meta-learners of stacked ensembles, a new weighted combination-based meta-learner that maximizes the H-measure during the training of stacked ensemble was first introduced and implemented in the empirical evaluation of this paper. The classification performances of stacked ensembles with 19 different meta-learners were recorded using both the area under the receiver operating characteristic curve (AUC) and H-measure (a metric that overcomes the deficiencies of the AUC). The weighted combination-based meta-learners of stacked ensembles have better classification performances on imbalanced datasets when compared to bagging-based, boosting-based, Decision Trees, Support Vector Machines, Naive Bayes, and Feedforward Neural Network meta-learners. Thus, the adoption of weighted combination-based meta-learners in stacked ensembles is recommended for their better performance on imbalanced datasets. Also, based on the empirical results, we identified better-performing meta-learners (such as the AUC maximizing meta-learner and the H-measure maximizing meta-learner) than the widely adopted meta-learner – Logistic Regression – in imbalanced classification.

In this paper, we propose a technique to recognize multiple actions in a video using deep learning. The proposed approach is concerned with interpreting the overall context of a video and transforming it into one or more appropriate actions. In order to cope with multiple actions in a video, our proposed technique first determines the individual segments/shots in a video using intersections of color histograms. The segmented parts are then fed to the action recognition system comprising a combination of a Convolution Neural Network (CNN) and a Long-Short-Term Memory (LSTM) network trained on our action vocabulary. The segments are then labeled according to their predicted actions and a compact set of distinct actions is produced. Using the corpus generated by the shot detection phase, which includes the location of keyframes in shots, and start/end timestamps of a shot, we can also perform video segmentation based on an action query. Hence, the proposed technique can be used for a number of tasks such as content censoring, on-demand scene retrieval, video summarizing, and query-based scene/video retrieval, to name a few. The proposed technique also stands apart from the existing approaches which either do not take into account the motion information for action prediction or do not perform action-based video segmentation. The experimental results presented in this paper show that the proposed technique not only finds the complete set of actions present in the video but can also find all the relevant parts in a video based on an action query.

Motivation: The localization of objects in images is a longstanding objective within the field of image processing. Most current techniques are based on machine learning approaches, which typically require careful annotation of training samples in the form of expensive bounding box labels. The need for such large-scale annotation has only been exacerbated by the widespread adoption of deep learning techniques within the image processing community: deep learning is notoriously data-hungry. Method: In this work, we attack this problem directly by providing a new method for learning to localize objects with limited annotation: most training images can simply be annotated with their whole image labels (and no bounding box), with only a small fraction marked with bounding boxes. The training is driven by a novel loss function, which is a continuous relaxation of a well-defined discrete formulation of weakly supervised learning. Care is taken to ensure that the loss is numerically well-posed. Additionally, we propose a neural network architecture which accounts for both patch dependence, through the use of Conditional Random Field layers, and shift-invariance, through the inclusion of anti-aliasing filters. Results: We demonstrate our method on the task of localizing thoracic diseases in chest X-ray images, achieving state-of-the-art performance on the ChestX-ray14 dataset. We further show that with a modicum of additional effort our technique can be extended from object localization to object detection, attaining high quality results on the Kaggle RSNA Pneumonia Detection Challenge. Conclusion: The technique presented in this paper has the potential to enable high accuracy localization in regimes in which annotated data is either scarce or expensive to acquire. Future work will focus on applying the ideas presented in this paper to the realm of semantic segmentation.

Design defects affect project quality and hinder development and maintenance. Consequently, experts need to minimize these defects in software systems. A promising approach is to apply the concepts of refactoring at higher level of abstraction based on UML diagrams instead of code level. Unfortunately, we find in literature many defects that are described textually and there is no consensus on how to decide if a particular design violates model quality. Defects could be quantified as metrics based rules that represent a combination of software metrics. However, it is difficult to find manually the best threshold values for these metrics. In this paper, we propose a new approach to identify design defects at the model level using the ID3 decision tree algorithm. We aim to create a decision tree for each defect. We experimented our approach on four design defects: The Blob, Data class, Lazy class and Feature Envy defect, using 15 Object-Oriented metrics. The rules generated using decision tree give a very promising detection results for the four open source projects tested in this paper. In Lucene 1.4 project, we found that the precision is 67% for a recall of 100%. In general, the accuracy varies from 49%, reaching for Lucene 1.4 project 80%.