Search

The engineering challenge of rolling bearing condition monitoring has led to a large number of method developments over the past few years. Most commonly, vibration measurement data are used for fault diagnosis using machine learning algorithms. In current research, purely data-driven deep learning methods are becoming increasingly popular, aiming for accurate predictions of bearing faults without requiring bearing-specific domain knowledge. Opposing this trend in popularity, the present paper takes a more traditional approach, incorporating domain knowledge by evaluating a variety of feature engineering methods in combination with a random forest classifier. For a comprehensive feature engineering study, a total of 42 mathematical feature formulas are combined with the preprocessing methods of envelope analysis, empirical mode decomposition, wavelet transforms, and frequency band separations. While each single processing method and feature formula is known from the literature, the presented paper contributes to the body of knowledge by investigating novel series connections of processing methods and feature formulas. Using the CWRU bearing fault data for performance evaluation, feature calculation based on the processing method of frequency band separation leads to particularly high prediction accuracies, while at the same time being very efficient in terms of low computational effort. Additionally, in comparison with deep learning approaches, the proposed feature engineering method provides excellent accuracies and enables explainability. [ABSTRACT FROM AUTHOR]

Kicks can lead to well control risks during petroleum drilling, and even more serious kicks may lead to serious casualties, which is the biggest threat factor affecting the safety in the process of petroleum drilling. Therefore, how to detect kicks early and efficiently has become a focus practical problem. Traditional machine learning models require a large amount of labeled data, such kicked sample, and it is difficult to label data, which requires a lot of labor and time. To address the above issues, the deep forest is extended to a transfer learning model to improve the generalization ability. In this paper, a transfer learning model is built to detect kicks early. The source domain model adopts the deep forest model. Deep forest is an ensemble learning model with a hierarchical structure similar to deep learning. Each layer contains a variety of random forests. It is an integration of the model in depth and breadth. In the case of a small sample size (20–60 min), kick can be identified 10 min in advance. The deep forest model is established as the source domaining model, and a cascade forest is added at the last layer according to the transfer learning algorithm to form the classification model of this paper. The experimental results show that the kick prediction accuracy of the model is 80.13% by a confusion matrix. In the target domain, the proposed model performs better than other ensemble learning algorithms, and the accuracy is 5% lower than other SOTA transfer learning algorithms. [ABSTRACT FROM AUTHOR]

Defective wheels pose a significant challenge in railway transportation, impacting operational performance and safety. Excessive traction and braking forces give rise to deviations from the intended conical tread shape, resulting in amplified vibrations and noise. Moreover, these deviations contribute to the accelerated damage of track components. Detecting wheel defects at an early stage is crucial to ensure safe and comfortable operation, as well as to minimize maintenance costs. However, the presence of various vibrations, such as those induced by the track, traction motors, and other rolling stock subsystems, poses a significant challenge for onboard detection techniques. These vibrations create difficulties in accurately identifying wheel defects in real-time during operational activities, often resulting in false alarms. This research paper aims to address this issue by using a hybrid deep learning-based approach for the accurate detection of various types of wheel defects using accelerometer data. The proposed approach aims to enhance wheel defect detection accuracy while considering onboard techniques' cost-effectiveness and efficiency. A realistic simulation model of the railway wheelset is developed to generate a comprehensive dataset. To generate vibration data in various scenarios, the model is simulated for 20 s under different conditions, including one non-faulty scenario and six faulty scenarios. The simulations are conducted at different speeds and track conditions to capture a wide range of operating conditions. Within each simulation iteration, a total of 200,000 data points are generated, providing a comprehensive dataset for analysis and evaluation. The generated data are then utilized to train and evaluate a hybrid deep learning model, employing a multi-layer perceptron (MLP) as a feature extractor and multiple machine learning models (support vector machine, random forest, decision tree, and k-nearest neighbors) for performance comparison. The results demonstrate that the MLP-RF (multi-layer perceptron with random forest) model achieved an accuracy of 99%, while the MLP-DT (multi-layer perceptron with decision tree) model achieved an accuracy of 98%. These high accuracy values indicate the effectiveness of the models in accurately classifying and predicting the outcomes. The contributions of this research work include the development of a realistic simulation model, the evaluation of sensor layout effectiveness, and the application of deep learning techniques for improved wheel flat detections. [ABSTRACT FROM AUTHOR]

Particulate matter (PM) in the air can cause various health problems and diseases in humans. In particular, the smaller size of PM 2.5 enable them to penetrate deep into the lungs, causing severe health impacts. Exposure to PM 2.5 can result in respiratory, cardiovascular, and allergic diseases, and prolonged exposure has also been linked to an increased risk of cancer, including lung cancer. Therefore, forecasting the PM 2.5 concentration in the surrounding is crucial for preventing these adverse health effects. This paper proposes a method for forecasting the PM 2.5 concentration after 1 h using bidirectional long short-term memory (Bi-LSTM). The proposed method involves selecting input variables based on the feature importance calculated by random forest, classifying the data to assign weight variables to reduce bias, and forecasting the PM 2.5 concentration using Bi-LSTM. To compare the performance of the proposed method, two case studies were conducted. First, a comparison of forecasting performance according to preprocessing. Second, forecasting performance between deep learning (long short-term memory, gated recurrent unit, and Bi-LSTM) and conventional machine learning models (multi-layer perceptron, support vector machine, decision tree, and random forest). In case study 1, The proposed method shows that the performance indices (RMSE: 3.98%p, MAE: 5.87%p, RRMSE: 3.96%p, and R 2 :0.72%p) are improved because weights are given according to the input variables before the forecasting is performed. In case study 2, we show that Bi-LSTM, which considers both directions (forward and backward), can effectively forecast when compared to conventional models (RMSE: 2.70, MAE: 0.84, RRMSE: 1.97, R 2 : 0.16). Therefore, it is shown that the proposed method can effectively forecast PM 2.5 even if the data in the high-concentration section is insufficient. [ABSTRACT FROM AUTHOR]

The prediction of drug-target interactions (DTIs) is a vital step in drug discovery. The success of machine learning and deep learning methods in accurately predicting DTIs plays a huge role in drug discovery. However, when dealing with learning algorithms, the datasets used are usually highly dimensional and extremely imbalanced. To solve this issue, the dataset must be resampled accordingly. In this paper, we have compared several data resampling techniques to overcome class imbalance in machine learning methods as well as to study the effectiveness of deep learning methods in overcoming class imbalance in DTI prediction in terms of binary classification using ten (10) cancer-related activity classes from BindingDB. It is found that the use of Random Undersampling (RUS) in predicting DTIs severely affects the performance of a model, especially when the dataset is highly imbalanced, thus, rendering RUS unreliable. It is also found that SVM-SMOTE can be used as a go-to resampling method when paired with the Random Forest and Gaussian Naïve Bayes classifiers, whereby a high F1 score is recorded for all activity classes that are severely and moderately imbalanced. Additionally, the deep learning method called Multilayer Perceptron recorded high F1 scores for all activity classes even when no resampling method was applied. [ABSTRACT FROM AUTHOR]

In this paper, a methodology based on data-driven models is developed to predict the NOx emissions of an internal combustion engine using, as inputs, a set of ECU channels representing the main engine actuations. Several regressors derived from the machine learning and deep learning algorithms are tested and compared in terms of prediction accuracy and computational efficiency to assess the most suitable for the aim of this work. Six Real Driving Emission (RDE) cycles performed at the roll bench were used for the model training, while another two RDE cycles and a steady-state map of NOx emissions were used to test the model under dynamic and stationary conditions, respectively. The models considered include Polynomial Regressor (PR), Support Vector Regressor (SVR), Random Forest Regressor (RF), Light Gradient Boosting Regressor (LightGBR) and Feed-Forward Neural Network (ANN). Ensemble methods such as Random Forest and LightGBR proved to have similar performances in terms of prediction accuracy, with LightGBR requiring a much lower training time. Afterwards, LightGBR predictions are compared with experimental NOx measurements in steady-state conditions and during two RDE cycles. Coefficient of determination (R2), normalized root mean squared error (nRMSE) and mean average percentage error (MAPE) are the main metrics used. The NOx emissions predicted by the LightGBR show good coherence with the experimental test set, both with the steady-state NOx map (R2 = 0.91 and MAPE = 6.42%) and with the RDE cycles (R2 = 0.95 and nRMSE = 0.04). [ABSTRACT FROM AUTHOR]

It is significant for anesthesiologists to have a precise grasp of the recovery time of the patient after anesthesia. Accurate prediction of anesthesia recovery time can support anesthesiologist decision-making during surgery to help reduce the risk of surgery in patients. However, effective models are not proposed to solve this problem for anesthesiologists. In this paper, we seek to find effective forecasting methods. First, we collect 1824 patient anesthesia data from the eye center and then performed data preprocessing. We extracted 85 variables to predict recovery time from anesthesia. Second, we extract anesthesia information between variables for prediction using machine learning methods, including Bayesian ridge, lightGBM, random forest, support vector regression, and extreme gradient boosting. We also design simple deep learning models as prediction models, including linear residual neural networks and jumping knowledge linear neural networks. Lastly, we perform a comparative experiment of the above methods on the dataset. The experiment demonstrates that the machine learning method performs better than the deep learning model mentioned above on a small number of samples. We find random forest and XGBoost are more efficient than other methods to extract information between variables on postoperative anesthesia recovery time. [ABSTRACT FROM AUTHOR]

In predicting energy consumption, classic econometric and statistical models are used to forecast energy consumption. These models may have limitations in an increasingly fast-changing energy market, which requires big data analysis of energy consumption patterns and relevant variables using complex mathematical tools. In current literature, there are minimal comparison studies reviewing machine learning algorithms to predict energy consumption in Korea. To bridge this gap, this paper compared three different machine learning algorithms, namely the Random Forest (RF) model, XGBoost (XGB) model, and Long Short-Term Memory (LSTM) model. These algorithms were applied in Period 1 (prior to the onset of the COVID-19 pandemic) and Period 2 (after the onset of the COVID-19 pandemic). Period 1 was characterized by an upward trend in energy consumption, while Period 2 showed a reduction in energy consumption. LSTM performed best in its prediction power specifically in Period 1, and RF outperformed the other models in Period 2. Findings, therefore, suggested the applicability of machine learning to forecast energy consumption and also demonstrated that traditional econometric approaches may outperform machine learning when there is less unknown irregularity in the time series, but machine learning can work better with unexpected irregular time series data. [ABSTRACT FROM AUTHOR]

In aerospace, marine, and other heavy industries, bearing fault diagnosis has been an essential part of improving machine life, reducing economic losses, and avoiding safety problems caused by machine bearing failures. Most existing bearing fault diagnosis methods face challenges in extracting the fault features from raw bearing fault data. Compared with traditional methods for bearing fault characteristics extraction, deep neural networks can automatically extract intrinsic features without expert knowledge. The convolutional neural network (CNN) was utilized most widely in extracting representative features of bearing faults. Fundamental to this, the hybrid models based on the CNN and individual classifiers were proposed to diagnose bearing faults. However, CNN may not be suitable for all bearing fault classifiers. It is crucial to identify the classifiers which can maximize the CNN feature extraction ability. In this paper, four hybrid models based on CNN were built, and their fault detection accuracy and efficiency were compared. The comparative analysis showed that the random forest (RF) and support vector machine (SVM) could make full use of the CNN feature extraction ability. [ABSTRACT FROM AUTHOR]

The vibration signals for offshore wind-turbine high-speed bearings are often contaminated with noises due to complex environmental and structural loads, which increase the difficulty of fault detection and diagnosis. In view of this problem, we propose a fault-diagnosis strategy with good noise immunity in this paper by integrating the two-dimensional convolutional neural network (2DCNN) with random forest (RF), which is supposed to utilize both CNN's automatic feature-extraction capability and the robust discrimination performance of RF classifiers. More specifically, the raw 1D time-domain bearing-vibration signals are transformed into 2D grayscale images at first, which are then fed to the 2DCNN-RF model for fault diagnosis. At the same time, three procedures, including exponential linear unit (ELU), batch normalization (BN), and dropout, are introduced in the model to improve feature-extraction performance and the noise immune capability. In addition, when the 2DCNN feature extractor is trained, the obtained feature vectors are passed to the RF classifier to improve the classification accuracy and generalization ability of the model. The experimental results show that the diagnostic accuracy of the 2DCNN-RF model could achieve 99.548% on the CWRU high-speed bearing dataset, which outperforms the standard CNN and other standard machine-learning and deep-learning algorithms. Furthermore, when the vibration signals are polluted with noises, the 2DCNN-RF model, without retraining the model or any denoising process, still achieves satisfying performance with higher accuracy than the other methods. [ABSTRACT FROM AUTHOR]

Eddy covariance observation is an applicable way to obtain accurate and continuous carbon flux at flux tower sites, while remote sensing technology could estimate carbon exchange and carbon storage at regional and global scales effectively. However, it is still challenging to up-scale the field-observed carbon flux to a regional scale, due to the heterogeneity and the unstable air conditions at the land surface. In this paper, gross primary production (GPP) from ground eddy covariance systems were up-scaled to a regional scale by using five machine learning methods (Cubist regression tree, random forest, support vector machine, artificial neural network, and deep belief network). Then, the up-scaled GPP were validated using GPP at flux tower sites, weighted GPP in the footprint, and MODIS GPP products. At last, the sensitivity of the input data (normalized difference vegetation index, fractional vegetation cover, shortwave radiation, relative humidity and air temperature) to the precision of up-scaled GPP was analyzed, and the uncertainty of the machine learning methods was discussed. The results of this paper indicated that machine learning methods had a great potential in up-scaling GPP at flux tower sites. The validation of up-scaled GPP, using five machine learning methods, demonstrated that up-scaled GPP using random forest obtained the highest accuracy. [ABSTRACT FROM AUTHOR]

The occurrence of rockburst can cause significant disasters in underground rock engineering. It is crucial to predict and prevent rockburst in deep tunnels and mines. In this paper, the deficiencies of ensemble learning algorithms in rockburst prediction were investigated. Aiming at these shortages, a novel machine learning model, deep forest, was proposed to predict rockburst risk. The deep forest combines the characteristics of deep learning and ensemble models, which can solve complex problems. To develop the deep forest model for rockburst prediction, 329 real rockburst cases were collected to build a comprehensive database for intelligent analysis. Bayesian optimization was proposed to tune the hyperparameters of the deep forest. As a result, the deep forest model achieved 100% training accuracy and 92.4% testing accuracy, and it has more outstanding capability to forecast rockburst disasters compared to other widely used models (i.e., random forest, boosting tree models, neural network, support vector machine, etc.). The results of sensitivity analysis revealed the impact of variables on rockburst levels and the applicability of deep forest with a few input parameters. Eventually, real cases of rockburst in two gold mines, China, were used for validation purposes while the needed data sets were prepared by field observations and laboratory tests. The promoting results of the developed model during the validation phase confirm that it can be used with a high level of accuracy by practicing engineers for predicting rockburst occurrences. [ABSTRACT FROM AUTHOR]

This paper presents a computer-vision-based methodology for automatic image-based classification of 2042 training images and 284 unseen (test) images divided into 68 categories of gemstones. A series of feature extraction techniques (33 including colour histograms in the RGB, HSV and CIELAB space, local binary pattern, Haralick texture and grey-level co-occurrence matrix properties) were used in combination with different machine-learning algorithms (Logistic Regression, Linear Discriminant Analysis, K-Nearest Neighbour, Decision Tree, Random Forest, Naive Bayes and Support Vector Machine). Deep-learning classification with ResNet-18 and ResNet-50 was also investigated. The optimal combination was provided by a Random Forest algorithm with the RGB eight-bin colour histogram and local binary pattern features, with an accuracy of 69.4% on unseen images; the algorithms required 0.0165 s to process the 284 test images. These results were compared against three expert gemmologists with at least 5 years of experience in gemstone identification, who obtained accuracies between 42.6% and 66.9% and took 42–175 min to classify the test images. As expected, the human experts took much longer than the computer vision algorithms, which in addition provided, albeit marginal, higher accuracy. Although these experiments included a relatively low number of images, the superiority of computer vision over humans is in line with what has been reported in other areas of study, and it is encouraging to further explore the application in gemmology and related areas. [ABSTRACT FROM AUTHOR]

Accurate forecasting of future meteorological elements is critical and has profoundly affected human life in many aspects from rainstorm warning to flight safety. The conventional numerical weather prediction (NWP) sometimes leads to unsatisfactory performance due to inappropriate initial state settings. In this paper, a short-term weather forecasting model based on wavelet packet denoising and Catboost is proposed, which takes advantage of the fusion information combining the historical observation data with the prior knowledge from NWP. The feature selection and spatiotemporal feather addition are also explored to further improve performance. The proposed method is evaluated on the datasets provided by Beijing weather stations. Experimental results demonstrate that compared with many deep-learning or machine-learning methods such as LSTM, Seq2Seq, and random forest, the proposed Catboost model incorporated with wavelet packet denoising can achieve shorter convergence time and higher prediction accuracy. [ABSTRACT FROM AUTHOR]

Existing accelerometer-based human activity recognition (HAR) benchmark datasets that were recorded during free living suffer from non-fixed sensor placement, the usage of only one sensor, and unreliable annotations. We make two contributions in this work. First, we present the publicly available Human Activity Recognition Trondheim dataset (HARTH). Twenty-two participants were recorded for 90 to 120 min during their regular working hours using two three-axial accelerometers, attached to the thigh and lower back, and a chest-mounted camera. Experts annotated the data independently using the camera's video signal and achieved high inter-rater agreement (Fleiss' Kappa = 0.96 ). They labeled twelve activities. The second contribution of this paper is the training of seven different baseline machine learning models for HAR on our dataset. We used a support vector machine, k-nearest neighbor, random forest, extreme gradient boost, convolutional neural network, bidirectional long short-term memory, and convolutional neural network with multi-resolution blocks. The support vector machine achieved the best results with an F1-score of 0.81 (standard deviation: ± 0.18 ), recall of 0.85 ± 0.13 , and precision of 0.79 ± 0.22 in a leave-one-subject-out cross-validation. Our highly professional recordings and annotations provide a promising benchmark dataset for researchers to develop innovative machine learning approaches for precise HAR in free living. [ABSTRACT FROM AUTHOR]

Many smart city and society applications such as smart health (elderly care, medical applications), smart surveillance, sports, and robotics require the recognition of user activities, an important class of problems known as human activity recognition (HAR). Several issues have hindered progress in HAR research, particularly due to the emergence of fog and edge computing, which brings many new opportunities (a low latency, dynamic and real-time decision making, etc.) but comes with its challenges. This paper focuses on addressing two important research gaps in HAR research: (i) improving the HAR prediction accuracy and (ii) managing the frequent changes in the environment and data related to user activities. To address this, we propose an HAR method based on Soft-Voting and Self-Learning (SVSL). SVSL uses two strategies. First, to enhance accuracy, it combines the capabilities of Deep Learning (DL), Generalized Linear Model (GLM), Random Forest (RF), and AdaBoost classifiers using soft-voting. Second, to classify the most challenging data instances, the SVSL method is equipped with a self-training mechanism that generates training data and retrains itself. We investigate the performance of our proposed SVSL method using two publicly available datasets on six human activities related to lying, sitting, and walking positions. The first dataset consists of 562 features and the second dataset consists of five features. The data are collected using the accelerometer and gyroscope smartphone sensors. The results show that the proposed method provides 6.26%, 1.75%, 1.51%, and 4.40% better prediction accuracy (average over the two datasets) compared to GLM, DL, RF, and AdaBoost, respectively. We also analyze and compare the class-wise performance of the SVSL methods with that of DL, GLM, RF, and AdaBoost. [ABSTRACT FROM AUTHOR]

With the development of high-throughput technologies, more and more high-dimensional or ultra-high-dimensional genomic data are being generated. Therefore, effectively analyzing such data has become a significant challenge. Machine learning (ML) algorithms have been widely applied for modeling nonlinear and complicated interactions in a variety of practical fields such as high-dimensional survival data. Recently, multilayer deep neural network (DNN) models have made remarkable achievements. Thus, a Cox-based DNN prediction survival model (DNNSurv model), which was built with Keras and TensorFlow, was developed. However, its results were only evaluated on the survival datasets with high-dimensional or large sample sizes. In this paper, we evaluated the prediction performance of the DNNSurv model using ultra-high-dimensional and high-dimensional survival datasets and compared it with three popular ML survival prediction models (i.e., random survival forest and the Cox-based LASSO and Ridge models). For this purpose, we also present the optimal setting of several hyperparameters, including the selection of a tuning parameter. The proposed method demonstrated via data analysis that the DNNSurv model performed well overall as compared with the ML models, in terms of the three main evaluation measures (i.e., concordance index, time-dependent Brier score, and the time-dependent AUC) for survival prediction performance. [ABSTRACT FROM AUTHOR]

The deep random forest (DRF) has recently gained new attention in deep learning because it has a high performance similar to that of a deep neural network (DNN) and does not rely on a backpropagation. However, it connects a large number of decision trees to multiple layers, thereby making analysis difficult. This paper proposes a new method for simplifying a black-box model of a DRF using a proposed rule elimination. For this, we consider quantifying the feature contributions and frequency of the fully trained DRF in the form of a decision rule set. The feature contributions provide a basis for determining how features affect the decision process in a rule set. Model simplification is achieved by eliminating unnecessary rules by measuring the feature contributions. Consequently, the simplified and transparent DRF has fewer parameters and rules than before. The proposed method was successfully applied to various DRF models and benchmark sensor datasets while maintaining a robust performance despite the elimination of a large number of rules. A comparison with state-of-the-art compressed DNNs also showed the proposed model simplification's higher parameter compression and memory efficiency with a similar classification accuracy. [ABSTRACT FROM AUTHOR]

Background: the credit scoring model is an effective tool for banks and other financial institutions to distinguish potential default borrowers. The credit scoring model represented by machine learning methods such as deep learning performs well in terms of the accuracy of default discrimination, but the model itself also has many shortcomings such as many hyperparameters and large dependence on big data. There is still a lot of room to improve its interpretability and robustness. Methods: the deep forest or multi-Grained Cascade Forest (gcForest) is a decision tree depth model based on the random forest algorithm. Using multidimensional scanning and cascading processing, gcForest can effectively identify and process high-dimensional feature information. At the same time, gcForest has fewer hyperparameters and has strong robustness. So, this paper constructs a two-stage hybrid default discrimination model based on multiple feature selection methods and gcForest algorithm, and at the same time, it optimizes the parameters for the lowest type II error as the first principle, and the highest AUC and accuracy as the second and third principles. GcForest can not only reflect the advantages of traditional statistical models in terms of interpretability and robustness but also take into account the advantages of deep learning models in terms of accuracy. Results: the validity of the hybrid default discrimination model is verified by three real open credit data sets of Australian, Japanese, and German in the UCI database. Conclusions: the performance of the gcForest is better than the current popular single classifiers such as ANN, and the common ensemble classifiers such as LightGBM, and CNNs in type II error, AUC, and accuracy. Besides, in comparison with other similar research results, the robustness and effectiveness of this model are further verified. [ABSTRACT FROM AUTHOR]