Search

This paper proposes an advanced deep learning model that integrates the Diffusion-Transformer structure and parallel attention mechanism for the tasks of growth estimation and disease detection in jujube forests. Existing methods in forestry monitoring often fall short in meeting the practical needs of large-scale and highly complex forest areas due to limitations in data processing capabilities and feature extraction precision. In response to this challenge, this paper designs and conducts a series of benchmark tests and ablation experiments to systematically evaluate and verify the performance of the proposed model across key performance metrics such as precision, recall, accuracy, and F1-score. Experimental results demonstrate that compared to traditional machine learning models like Support Vector Machines and Random Forests, as well as common deep learning models such as AlexNet and ResNet, the model proposed in this paper achieves a precision of 95%, a recall of 92%, an accuracy of 93%, and an F1-score of 94% in the task of disease detection in jujube forests, showing similarly superior performance in growth estimation tasks as well. Furthermore, ablation experiments with different attention mechanisms and loss functions further validate the effectiveness of parallel attention and parallel loss function in enhancing the overall performance of the model. These research findings not only provide a new technical path for forestry disease monitoring and health assessment but also contribute rich theoretical and experimental foundations for related fields. [ABSTRACT FROM AUTHOR]

Given the increasing competition among household appliance enterprises, accurately predicting household appliance demand is crucial for enterprise supply chain management and marketing. This paper proposes a combined model integrating deep learning and ensemble learning—LSTM-RF-XGBoost—to assist enterprises in identifying customer demand, thereby addressing the complexity and uncertainty of the household appliance market demand. In this study, Long Short-Term Memory Network (LSTM), Random Forest (RF), and Extreme Gradient Boosting (XGBoost) models are established separately. Then, the three individual algorithms are used as the base models in the first layer, with the multiple linear regression (MLR) algorithm serving as the meta-model in the second layer, merging the demand prediction model based on the hybrid model into the overall demand prediction model. This study demonstrates that the accuracy and stability of demand prediction using the LSTM–RF–XGBoost model significantly outperform traditional single models, highlighting the significant advantages of using a combined model. This research offers practical and innovative solutions for enterprises seeking rational resource allocation through demand prediction. [ABSTRACT FROM AUTHOR]

Smart contracts are programs that reside and execute on a blockchain, like any transaction. They are automatically executed when preprogrammed terms and conditions are met. Although the smart contract (SC) must be presented in the blockchain for the integrity of data and transactions stored within it, it is highly exposed to several vulnerabilities attackers exploit to access the data. In this paper, classification and detection of vulnerabilities targeting smart contracts are performed using deep learning algorithms over two datasets containing 12,253 smart contracts. These contracts are converted into RGB and Grayscale images and then inserted into Residual Network (ResNet50), Visual Geometry Group-19 (VGG19), Dense Convolutional Network (DenseNet201), k-nearest Neighbors (KNN), and Random Forest (RF) algorithms for binary and multi-label classification. A comprehensive analysis is conducted to detect and classify vulnerabilities using different performance metrics. The performance of these algorithms was outstanding, accurately classifying vulnerabilities with high F1 scores and accuracy rates. For binary classification, RF emerged in RGB images as the best algorithm based on the highest F1 score of 86.66% and accuracy of 86.66%. Moving on to multi-label classification, VGG19 stood out in RGB images as the standout algorithm, achieving an impressive accuracy of 89.14% and an F1 score of 85.87%. To the best of our knowledge, and according to the available literature, this study is the first to investigate binary classification of vulnerabilities targeting Ethereum smart contracts, and the experimental results of the proposed methodology for multi-label vulnerability classification outperform existing literature. [ABSTRACT FROM AUTHOR]

Site-specific weed management employs image data to generate maps through various methodologies that classify pixels corresponding to crop, soil, and weed. Further, many studies have focused on identifying specific weed species using spectral data. Nonetheless, the availability of open-access weed datasets remains limited. Remarkably, despite the extensive research employing hyperspectral imaging data to classify species under varying conditions, to the best of our knowledge, there are no open-access hyperspectral weed datasets. Consequently, accessible spectral weed datasets are primarily RGB or multispectral and mostly lack the temporal aspect, i.e., they contain a single measurement day. This paper introduces an open dataset for training and evaluating machine-learning methods and spectral features to classify weeds based on various biological traits. The dataset comprises 30 hyperspectral images, each containing thousands of pixels with 204 unique visible and near-infrared bands captured in a controlled environment. In addition, each scene includes a corresponding RGB image with a higher spatial resolution. We included three weed species in this dataset, representing different botanical groups and photosynthetic mechanisms. In addition, the dataset contains meticulously sampled labeled data for training and testing. The images represent a time series of the weed's growth along its early stages, critical for precise herbicide application. We conducted an experimental evaluation to test the performance of a machine-learning approach, a deep-learning approach, and Spectral Mixture Analysis (SMA) to identify the different weed traits. In addition, we analyzed the importance of features using the random forest algorithm and evaluated the performance of the selected algorithms while using different sets of features. [ABSTRACT FROM AUTHOR]

Cardiovascular disease (CVD) is a leading cause of death globally; therefore, early detection of CVD is crucial. Many intelligent technologies, including deep learning and machine learning (ML), are being integrated into healthcare systems for disease prediction. This paper uses a voting ensemble ML with chi-square feature selection to detect CVD early. Our approach involved applying multiple ML classifiers, including naïve Bayes, random forest, logistic regression (LR), and k-nearest neighbor. These classifiers were evaluated through metrics including accuracy, specificity, sensitivity, F1-score, confusion matrix, and area under the curve (AUC). We created an ensemble model by combining predictions from the different ML classifiers through a voting mechanism, whose performance was then measured against individual classifiers. Furthermore, we applied chi-square feature selection method to the 303 records across 13 clinical features in the Cleveland cardiac disease dataset to identify the 5 most important features. This approach improved the overall accuracy of our ensemble model and reduced the computational load considerably by more than 50%. Demonstrating superior effectiveness, our voting ensemble model achieved a remarkable accuracy of 92.11%, representing an average improvement of 2.95% over the single highest classifier (LR). These results indicate the ensemble method as a viable and practical approach to improve the accuracy of CVD prediction. [ABSTRACT FROM AUTHOR]

The engineering challenge of rolling bearing condition monitoring has led to a large number of method developments over the past few years. Most commonly, vibration measurement data are used for fault diagnosis using machine learning algorithms. In current research, purely data-driven deep learning methods are becoming increasingly popular, aiming for accurate predictions of bearing faults without requiring bearing-specific domain knowledge. Opposing this trend in popularity, the present paper takes a more traditional approach, incorporating domain knowledge by evaluating a variety of feature engineering methods in combination with a random forest classifier. For a comprehensive feature engineering study, a total of 42 mathematical feature formulas are combined with the preprocessing methods of envelope analysis, empirical mode decomposition, wavelet transforms, and frequency band separations. While each single processing method and feature formula is known from the literature, the presented paper contributes to the body of knowledge by investigating novel series connections of processing methods and feature formulas. Using the CWRU bearing fault data for performance evaluation, feature calculation based on the processing method of frequency band separation leads to particularly high prediction accuracies, while at the same time being very efficient in terms of low computational effort. Additionally, in comparison with deep learning approaches, the proposed feature engineering method provides excellent accuracies and enables explainability. [ABSTRACT FROM AUTHOR]

Kicks can lead to well control risks during petroleum drilling, and even more serious kicks may lead to serious casualties, which is the biggest threat factor affecting the safety in the process of petroleum drilling. Therefore, how to detect kicks early and efficiently has become a focus practical problem. Traditional machine learning models require a large amount of labeled data, such kicked sample, and it is difficult to label data, which requires a lot of labor and time. To address the above issues, the deep forest is extended to a transfer learning model to improve the generalization ability. In this paper, a transfer learning model is built to detect kicks early. The source domain model adopts the deep forest model. Deep forest is an ensemble learning model with a hierarchical structure similar to deep learning. Each layer contains a variety of random forests. It is an integration of the model in depth and breadth. In the case of a small sample size (20–60 min), kick can be identified 10 min in advance. The deep forest model is established as the source domaining model, and a cascade forest is added at the last layer according to the transfer learning algorithm to form the classification model of this paper. The experimental results show that the kick prediction accuracy of the model is 80.13% by a confusion matrix. In the target domain, the proposed model performs better than other ensemble learning algorithms, and the accuracy is 5% lower than other SOTA transfer learning algorithms. [ABSTRACT FROM AUTHOR]

Experimental results reveal that rocking shallow foundations reduce earthquake-induced force and flexural displacement demands transmitted to structures and can be used as an effective geotechnical seismic isolation mechanism. This paper presents data-driven predictive models for maximum acceleration transmitted to structures founded on rocking shallow foundations during earthquake loading. Results from base-shaking experiments on rocking foundations have been utilized for the development of artificial neural network regression (ANN), k-nearest neighbors regression, support vector regression, random forest regression, adaptive boosting regression, and gradient boosting regression models. Acceleration amplification ratio, defined as the maximum acceleration at the center of gravity of a structure divided by the peak ground acceleration of the earthquake, is considered as the prediction parameter. For five out of six models developed in this study, the overall mean absolute percentage error in predictions in repeated k-fold cross validation tests vary between 0.128 and 0.145, with the ANN model being the most accurate and most consistent. The cross validation mean absolute error in predictions of all six models vary between 0.08 and 0.1, indicating that the maximum acceleration of structures supported by rocking foundations can be predicted within an average error limit of 8% to 10% of the peak ground acceleration of the earthquake. [ABSTRACT FROM AUTHOR]

Student engagement is a flexible, complicated concept that includes behavioural, emotional, and cognitive involvement. In order for the instructor to understand how the student interacts with the various activities in the classroom, it is essential to predict their participation. The current work aims to identify the best algorithm for predicting student engagement in the classroom. In this paper, we gathered data from VLE and prepared them using a variety of data preprocessing techniques, including the elimination of missing values, normalization, encoding, and identification of outliers. On our data, we ran a number of machine learning (ML) classification algorithms, and we assessed each one using cross-validation methods and many helpful indicators. The performance of the model is evaluated with metrics like accuracy, precision, recall, and AUC scores. The results show that the CATBoost model is having higher accuracy than the rest. This proposed model outperformed in all the aspects compared to previous research. The results part of this paper indicates that the CATBoost model had an accuracy of approximately 92.23%, a precision of 94.40%, a recall of 100%, and an AUC score of 0.9624. The XGBoost predictive model, the random forest model, and the multilayer perceptron model all demonstrated approximately the same performance overall. We compared the AISAR model with Our model achieved an accuracy of 94.64% compared with AISAR 91% model and it concludes that our results are better. The AISAR model had only around 50% recall compared to our models, which had around 92%. This shows that our models return more relevant results, i.e., if our models predict that a student has high engagement, they are correct 94.64% of the time. [ABSTRACT FROM AUTHOR]

Software-defined networking (SDN) is becoming the standard for the management of networks due to its scalability and flexibility to program the network. SDN provides many advantages but it also involves some specific security problems; for example, the controller can be taken down using cyber attacks, which can result in the whole network shutting down, creating a single point of failure. In this paper, DDoS attacks in SDN are detected using AI-enabled machine and deep learning models with some specific features for a dataset under normal DDoS traffic. In our approach, the initial dataset is collected from 84 features on Kaggle and then the 20 top features are selected using a permutation importance algorithm. The dataset is learned and tested with five AI-enabled models. Our experimental results show that the use of a machine learning-based random forest model achieves the highest accuracy rate of 99.97% in DDoS attack detection in SDN. Our contributions through this study are, firstly, that we found the top 20 features that contributed to DDoS attacks. Secondly, we reduce the time and cost of comparing various learning models and their performance in determining a learning model suitable for DDoS detection. Finally, various experimental methods to evaluate the performance of the learning model are presented so that related researchers can utilize them. [ABSTRACT FROM AUTHOR]

Defective wheels pose a significant challenge in railway transportation, impacting operational performance and safety. Excessive traction and braking forces give rise to deviations from the intended conical tread shape, resulting in amplified vibrations and noise. Moreover, these deviations contribute to the accelerated damage of track components. Detecting wheel defects at an early stage is crucial to ensure safe and comfortable operation, as well as to minimize maintenance costs. However, the presence of various vibrations, such as those induced by the track, traction motors, and other rolling stock subsystems, poses a significant challenge for onboard detection techniques. These vibrations create difficulties in accurately identifying wheel defects in real-time during operational activities, often resulting in false alarms. This research paper aims to address this issue by using a hybrid deep learning-based approach for the accurate detection of various types of wheel defects using accelerometer data. The proposed approach aims to enhance wheel defect detection accuracy while considering onboard techniques' cost-effectiveness and efficiency. A realistic simulation model of the railway wheelset is developed to generate a comprehensive dataset. To generate vibration data in various scenarios, the model is simulated for 20 s under different conditions, including one non-faulty scenario and six faulty scenarios. The simulations are conducted at different speeds and track conditions to capture a wide range of operating conditions. Within each simulation iteration, a total of 200,000 data points are generated, providing a comprehensive dataset for analysis and evaluation. The generated data are then utilized to train and evaluate a hybrid deep learning model, employing a multi-layer perceptron (MLP) as a feature extractor and multiple machine learning models (support vector machine, random forest, decision tree, and k-nearest neighbors) for performance comparison. The results demonstrate that the MLP-RF (multi-layer perceptron with random forest) model achieved an accuracy of 99%, while the MLP-DT (multi-layer perceptron with decision tree) model achieved an accuracy of 98%. These high accuracy values indicate the effectiveness of the models in accurately classifying and predicting the outcomes. The contributions of this research work include the development of a realistic simulation model, the evaluation of sensor layout effectiveness, and the application of deep learning techniques for improved wheel flat detections. [ABSTRACT FROM AUTHOR]

Particulate matter (PM) in the air can cause various health problems and diseases in humans. In particular, the smaller size of PM 2.5 enable them to penetrate deep into the lungs, causing severe health impacts. Exposure to PM 2.5 can result in respiratory, cardiovascular, and allergic diseases, and prolonged exposure has also been linked to an increased risk of cancer, including lung cancer. Therefore, forecasting the PM 2.5 concentration in the surrounding is crucial for preventing these adverse health effects. This paper proposes a method for forecasting the PM 2.5 concentration after 1 h using bidirectional long short-term memory (Bi-LSTM). The proposed method involves selecting input variables based on the feature importance calculated by random forest, classifying the data to assign weight variables to reduce bias, and forecasting the PM 2.5 concentration using Bi-LSTM. To compare the performance of the proposed method, two case studies were conducted. First, a comparison of forecasting performance according to preprocessing. Second, forecasting performance between deep learning (long short-term memory, gated recurrent unit, and Bi-LSTM) and conventional machine learning models (multi-layer perceptron, support vector machine, decision tree, and random forest). In case study 1, The proposed method shows that the performance indices (RMSE: 3.98%p, MAE: 5.87%p, RRMSE: 3.96%p, and R 2 :0.72%p) are improved because weights are given according to the input variables before the forecasting is performed. In case study 2, we show that Bi-LSTM, which considers both directions (forward and backward), can effectively forecast when compared to conventional models (RMSE: 2.70, MAE: 0.84, RRMSE: 1.97, R 2 : 0.16). Therefore, it is shown that the proposed method can effectively forecast PM 2.5 even if the data in the high-concentration section is insufficient. [ABSTRACT FROM AUTHOR]

With the recent growth of the Internet, the volume of data has also increased. In particular, the increase in the amount of unstructured data makes it difficult to manage data. Classification is also needed in order to be able to use the data for various purposes. Since it is difficult to manually classify the ever-increasing volume data for the purpose of various types of analysis and evaluation, automatic classification methods are needed. In addition, the performance of imbalanced and multi-class classification is a challenging task. As the number of classes increases, so does the number of decision boundaries a learning algorithm has to solve. Therefore, in this paper, an improvement model is proposed using WordNet lexical ontology and BERT to perform deeper learning on the features of text, thereby improving the classification effect of the model. It was observed that classification success increased when using WordNet 11 general lexicographer files based on synthesis sets, syntactic categories, and logical groupings. WordNet was used for feature dimension reduction. In experimental studies, word embedding methods were used without dimension reduction. Afterwards, Random Forest (RF), Support Vector Machine (SVM) and Multi-Layer Perceptron (MLP) algorithms were employed to perform classification. These studies were then repeated with dimension reduction performed by WordNet. In addition to the machine learning model, experiments were also conducted with the pretrained BERT model with and without WordNet. The experimental results showed that, on an unstructured, seven-class, imbalanced dataset, the highest accuracy value of 93.77% was obtained when using our proposed model. [ABSTRACT FROM AUTHOR]

The Quick UDP Internet Connections (QUIC) protocol provides advantages over traditional TCP, but its encryption functionality reduces the visibility for operators into network traffic. Many studies deploy machine learning and deep learning algorithms on QUIC traffic classification. However, standalone machine learning models are subject to overfitting and poor predictability in complex network traffic environments. Deep learning on the other hand requires a huge dataset and intensive parameter fine-tuning. On the contrary, ensemble techniques provide reliability, better prediction, and robustness of the trained model, thereby reducing the chance of overfitting. In this paper, we approach the QUIC network traffic classification problem by utilizing five different ensemble machine learning techniques, namely: Random Forest, Extra Trees, Gradient Boosting Tree, Extreme Gradient Boosting Tree, and Light Gradient Boosting Model. We used the publicly available dataset with five different services such as Google Drive, YouTube, Google Docs, Google Search, and Google Music. The models were trained using a different number of features on different scenarios and evaluated using several performance metrics. The results show that Extreme Gradient Boosting Tree and Light Gradient Boosting Model outperform the other models and achieve one of the highest results among the state-of-the-art models found in the literature with a simpler model and features. [ABSTRACT FROM AUTHOR]

The Internet of Things (IoT) has become widely adopted in businesses, organizations, and daily lives. They are usually characterized by transferring and processing sensitive data. Attackers have exploited this prospect of IoT devices to compromise user data's integrity and confidentiality. Considering the dynamic nature of the attacks, artificial intelligence (AI)-based techniques incorporating machine learning (ML) are promising techniques for identifying such attacks. However, the dataset being utilized features engineering techniques, and the kind of classifiers play significant roles in how accurate AI-based predictions are. Therefore, for the IoT environment, there is a need to contribute more to this context by evaluating different AI-based techniques on datasets that effectively capture the environment's properties. In this paper, we evaluated various ML models with the consideration of both binary and multiclass classification models validated on a new dedicated IoT dataset. Moreover, we investigated the impact of different features engineering techniques including correlation analysis and information gain. The experimental work conducted on bagging, k-nearest neighbor (KNN), J48, random forest (RF), logistic regression (LR), and multi-layer perceptron (MLP) models revealed that RF achieved the highest performance across all experiment sets, with a receiver operating characteristic (ROC) of 99.9%. [ABSTRACT FROM AUTHOR]

As the most widely used storage device today, hard disks are efficient and convenient, but the damage incurred in the event of a failure can be very significant. Therefore, early warnings before hard disk failure, allowing the stored content to be backed up and transferred in advance, can reduce many losses. In recent years, an endless stream of research on the prediction of hard disk failure prediction has emerged. The detection accuracy of various methods, from basic machine learning models, such as decision trees and random forests, to deep learning methods, such as BP neural networks and recurrent neural networks, has also been improving. In this paper, based on the idea of blending ensemble learning, a novel failure prediction method combining machine learning algorithms and neural networks is proposed on the publicly available BackBlaze hard disk datasets. The failure prediction experiment is conducted only with S.M.A.R.T., that is, the learned characteristics collected by self-monitoring analysis and reporting technology, which are internally counted during the operation of the hard disk. The experimental results show that this ensemble learning model is able to outperform other independent models in terms of evaluation criterion based on the Matthews correlation coefficient. Additionally, through the experimental results on multiple types of hard disks, an ensemble learning model with high performance on most types of hard disks is found, which solves the problem of the low robustness and generalization of traditional machine learning methods and proves the effectiveness and high universality of this method. [ABSTRACT FROM AUTHOR]

The prediction of drug-target interactions (DTIs) is a vital step in drug discovery. The success of machine learning and deep learning methods in accurately predicting DTIs plays a huge role in drug discovery. However, when dealing with learning algorithms, the datasets used are usually highly dimensional and extremely imbalanced. To solve this issue, the dataset must be resampled accordingly. In this paper, we have compared several data resampling techniques to overcome class imbalance in machine learning methods as well as to study the effectiveness of deep learning methods in overcoming class imbalance in DTI prediction in terms of binary classification using ten (10) cancer-related activity classes from BindingDB. It is found that the use of Random Undersampling (RUS) in predicting DTIs severely affects the performance of a model, especially when the dataset is highly imbalanced, thus, rendering RUS unreliable. It is also found that SVM-SMOTE can be used as a go-to resampling method when paired with the Random Forest and Gaussian Naïve Bayes classifiers, whereby a high F1 score is recorded for all activity classes that are severely and moderately imbalanced. Additionally, the deep learning method called Multilayer Perceptron recorded high F1 scores for all activity classes even when no resampling method was applied. [ABSTRACT FROM AUTHOR]

Forecasting stock market prices is an exciting knowledge area for investors and traders. Successful predictions lead to high financial revenues and prevent investors from market risks. This paper proposes a novel hybrid stock prediction model that improves prediction accuracy. The proposed method consists of three main components, a noise-filtering technique, novel features, and machine learning-based prediction. We used a fully modified Hodrick–Prescott filter to smooth the historical stock price data by removing the cyclic component from the time series. We propose several new features for stock price prediction, including the return of firm, return open price, return close price, change in return open price, change in return close price, and volume per total. We investigate traditional and deep machine learning approaches for prediction. Support vector regression, auto-regressive integrated moving averages, and random forests are used for conventional machine learning. Deep learning techniques comprise long short-term memory and gated recurrent units. We performed several experiments with these machine learning algorithms. Our best model achieved a prediction accuracy of 70.88%, a root-mean-square error of 0.04, and an error rate of 0.1. [ABSTRACT FROM AUTHOR]

In this paper, a methodology based on data-driven models is developed to predict the NOx emissions of an internal combustion engine using, as inputs, a set of ECU channels representing the main engine actuations. Several regressors derived from the machine learning and deep learning algorithms are tested and compared in terms of prediction accuracy and computational efficiency to assess the most suitable for the aim of this work. Six Real Driving Emission (RDE) cycles performed at the roll bench were used for the model training, while another two RDE cycles and a steady-state map of NOx emissions were used to test the model under dynamic and stationary conditions, respectively. The models considered include Polynomial Regressor (PR), Support Vector Regressor (SVR), Random Forest Regressor (RF), Light Gradient Boosting Regressor (LightGBR) and Feed-Forward Neural Network (ANN). Ensemble methods such as Random Forest and LightGBR proved to have similar performances in terms of prediction accuracy, with LightGBR requiring a much lower training time. Afterwards, LightGBR predictions are compared with experimental NOx measurements in steady-state conditions and during two RDE cycles. Coefficient of determination (R2), normalized root mean squared error (nRMSE) and mean average percentage error (MAPE) are the main metrics used. The NOx emissions predicted by the LightGBR show good coherence with the experimental test set, both with the steady-state NOx map (R2 = 0.91 and MAPE = 6.42%) and with the RDE cycles (R2 = 0.95 and nRMSE = 0.04). [ABSTRACT FROM AUTHOR]

It is significant for anesthesiologists to have a precise grasp of the recovery time of the patient after anesthesia. Accurate prediction of anesthesia recovery time can support anesthesiologist decision-making during surgery to help reduce the risk of surgery in patients. However, effective models are not proposed to solve this problem for anesthesiologists. In this paper, we seek to find effective forecasting methods. First, we collect 1824 patient anesthesia data from the eye center and then performed data preprocessing. We extracted 85 variables to predict recovery time from anesthesia. Second, we extract anesthesia information between variables for prediction using machine learning methods, including Bayesian ridge, lightGBM, random forest, support vector regression, and extreme gradient boosting. We also design simple deep learning models as prediction models, including linear residual neural networks and jumping knowledge linear neural networks. Lastly, we perform a comparative experiment of the above methods on the dataset. The experiment demonstrates that the machine learning method performs better than the deep learning model mentioned above on a small number of samples. We find random forest and XGBoost are more efficient than other methods to extract information between variables on postoperative anesthesia recovery time. [ABSTRACT FROM AUTHOR]

In predicting energy consumption, classic econometric and statistical models are used to forecast energy consumption. These models may have limitations in an increasingly fast-changing energy market, which requires big data analysis of energy consumption patterns and relevant variables using complex mathematical tools. In current literature, there are minimal comparison studies reviewing machine learning algorithms to predict energy consumption in Korea. To bridge this gap, this paper compared three different machine learning algorithms, namely the Random Forest (RF) model, XGBoost (XGB) model, and Long Short-Term Memory (LSTM) model. These algorithms were applied in Period 1 (prior to the onset of the COVID-19 pandemic) and Period 2 (after the onset of the COVID-19 pandemic). Period 1 was characterized by an upward trend in energy consumption, while Period 2 showed a reduction in energy consumption. LSTM performed best in its prediction power specifically in Period 1, and RF outperformed the other models in Period 2. Findings, therefore, suggested the applicability of machine learning to forecast energy consumption and also demonstrated that traditional econometric approaches may outperform machine learning when there is less unknown irregularity in the time series, but machine learning can work better with unexpected irregular time series data. [ABSTRACT FROM AUTHOR]

In aerospace, marine, and other heavy industries, bearing fault diagnosis has been an essential part of improving machine life, reducing economic losses, and avoiding safety problems caused by machine bearing failures. Most existing bearing fault diagnosis methods face challenges in extracting the fault features from raw bearing fault data. Compared with traditional methods for bearing fault characteristics extraction, deep neural networks can automatically extract intrinsic features without expert knowledge. The convolutional neural network (CNN) was utilized most widely in extracting representative features of bearing faults. Fundamental to this, the hybrid models based on the CNN and individual classifiers were proposed to diagnose bearing faults. However, CNN may not be suitable for all bearing fault classifiers. It is crucial to identify the classifiers which can maximize the CNN feature extraction ability. In this paper, four hybrid models based on CNN were built, and their fault detection accuracy and efficiency were compared. The comparative analysis showed that the random forest (RF) and support vector machine (SVM) could make full use of the CNN feature extraction ability. [ABSTRACT FROM AUTHOR]

The vibration signals for offshore wind-turbine high-speed bearings are often contaminated with noises due to complex environmental and structural loads, which increase the difficulty of fault detection and diagnosis. In view of this problem, we propose a fault-diagnosis strategy with good noise immunity in this paper by integrating the two-dimensional convolutional neural network (2DCNN) with random forest (RF), which is supposed to utilize both CNN's automatic feature-extraction capability and the robust discrimination performance of RF classifiers. More specifically, the raw 1D time-domain bearing-vibration signals are transformed into 2D grayscale images at first, which are then fed to the 2DCNN-RF model for fault diagnosis. At the same time, three procedures, including exponential linear unit (ELU), batch normalization (BN), and dropout, are introduced in the model to improve feature-extraction performance and the noise immune capability. In addition, when the 2DCNN feature extractor is trained, the obtained feature vectors are passed to the RF classifier to improve the classification accuracy and generalization ability of the model. The experimental results show that the diagnostic accuracy of the 2DCNN-RF model could achieve 99.548% on the CWRU high-speed bearing dataset, which outperforms the standard CNN and other standard machine-learning and deep-learning algorithms. Furthermore, when the vibration signals are polluted with noises, the 2DCNN-RF model, without retraining the model or any denoising process, still achieves satisfying performance with higher accuracy than the other methods. [ABSTRACT FROM AUTHOR]

Within the context of Industry 4.0, quality assessment procedures using data-driven techniques are becoming more critical due to the generation of massive amounts of production data. In this paper, we address the detection of abnormal screw tightening processes, which is a key industrial task. Since labeling is costly, requiring a manual effort, we focus on unsupervised detection approaches. In particular, we assume a computationally light low-dimensional problem formulation based on angle–torque pairs. Our work is focused on two unsupervised machine learning (ML) algorithms: isolation forest (IForest) and a deep learning autoencoder (AE). Several computational experiments were held by assuming distinct datasets and a realistic rolling window evaluation procedure. First, we compared the two ML algorithms with two other methods, a local outlier factor method and a supervised Random Forest, on older data related with two production days collected in November 2020. Since competitive results were obtained, during a second stage, we further compared the AE and IForest methods by adopting a more recent and larger dataset (from February to March 2021, totaling 26.9 million observations and related to three distinct assembled products). Both anomaly detection methods obtained an excellent quality class discrimination (higher than 90%) under a realistic rolling window with several training and testing updates. Turning to the computational effort, the AE is much lighter than the IForest for training (around 2.7 times faster) and inference (requiring 3.0 times less computation). This AE property is valuable within this industrial domain since it tends to generate big data. Finally, using the anomaly detection estimates, we developed an interactive visualization tool that provides explainable artificial intelligence (XAI) knowledge for the human operators, helping them to better identify the angle–torque regions associated with screw tightening failures. [ABSTRACT FROM AUTHOR]

Cybersecurity has become one of the focuses of organisations. The number of cyberattacks keeps increasing as Internet usage continues to grow. As new types of cyberattacks continue to emerge, researchers focus on developing machine learning (ML)-based intrusion detection systems (IDS) to detect zero-day attacks. They usually remove some or all attack samples from the training dataset and only include them in the testing dataset when evaluating the performance. This method may detect unknown attacks; however, it does not reflect the long-term performance of the IDS as it only shows the changes in the type of attacks. In this work, we focused on evaluating the long-term performance of ML-based IDS. To achieve this goal, we proposed evaluating the ML-based IDS using a dataset created later than the training dataset. The proposed method can better assess the long-term performance as the testing dataset reflects the changes in the attack type and network infrastructure changes over time. We have implemented six of the most popular ML models, including decision tree (DT), random forest (RF), support vector machine (SVM), naïve Bayes (NB), artificial neural network (ANN), and deep neural network (DNN). These models are trained and tested with a pair of datasets with symmetrical classes. Our experiments using the CIC-IDS2017 and the CSE-CIC-IDS2018 datasets show that SVM and ANN are most resistant to overfitting. Our experiments also indicate that DT and RF suffer the most from overfitting, although they perform well on the training dataset. On the other hand, our experiments using the LUFlow dataset have shown that all models can perform well when the difference between the training and testing datasets is small. [ABSTRACT FROM AUTHOR]

Patient-centered communication (PCC) quality is critical to increasing the quality of patient-centered care. Based on the nationally representative data of the Health Information National Trends Survey (HINTS) 2019–2020 (N = 4593), this study combined four machine learning methods, namely, Generalized Linear Models (GLM), Random Forests (Random Forests), Deep Neural Networks (Deep Learning), and Gradient Boosting Machines (GBM), to identify important PCC predictors through variable importance metrics. Fifteen variables were identified as important predictors, involving multiple dimensions, such as individual sociodemographic characteristics, health-related factors, and individual living habits. Among them, four novel potential associated variables are included, an individual's level of verbal expression, exercise habits, etc., which significantly impacted respondents' perceived PCC quality. This study revealed the value of combining feature selection with machine learning approaches to identify broad variables that could enhance PCC prediction and clinical decision-making, influence future PCC prediction research, and improve patient-centered care. In the future, other easy-to-interpret models can be combined to conduct further research on the impact direction and mechanism of important predictors on PCC. [ABSTRACT FROM AUTHOR]

Skin cancer diagnosis greatly benefits from advanced machine learning techniques, particularly fine-tuned deep learning models. In our research, we explored the impact of traditional machine learning and fine-tuned deep learning approaches on prediction accuracy. Our findings reveal significant improvements in predictability and accuracy with fine-tuning, particularly evident in deep learning models. The CNN, SVM, and Random Forest Classifier achieved high accuracy. However, fine-tuned deep learning models such as EfficientNetB0, ResNet34, VGG16, Inception _v3, and DenseNet121 demonstrated superior performance. To ensure comparability, we fine-tuned these models by incorporating additional layers, including one flatten layer and three densely interconnected layers. These layers play a crucial role in enhancing model efficiency and performance. The flatten layer preprocesses multidimensional feature maps, facilitating efficient information flow, while subsequent dense layers refine feature representations, capturing intricate patterns and relationships within the data. Leveraging LeakyReLU activation functions in the dense layers mitigates the vanishing gradient problem and promotes stable training. Finally, the output dense layer with a sigmoid activation function simplifies decision making for healthcare professionals by providing binary classification output. Our study underscores the significance of incorporating additional layers in fine-tuned neural network models for skin cancer classification, offering improved accuracy and reliability in diagnosis. [ABSTRACT FROM AUTHOR]

Tree species classification using hyperspectral imagery shows incredible promise in developing a large-scale, high-resolution model for identifying tree species, providing unprecedented details on global tree species distribution. Many questions remain unanswered about the best practices for creating a global, general hyperspectral tree species classification model. This study aims to address three key issues in creating a hyperspectral species classification model. We assessed the effectiveness of three data-labeling methods to create training data, three data-splitting methods for training/validation/testing, and machine-learning and deep-learning (including semi-supervised deep-learning) models for tree species classification using hyperspectral imagery at National Ecological Observatory Network (NEON) Sites. Our analysis revealed that the existing data-labeling method using the field vegetation structure survey performed reasonably well. The random tree data-splitting technique was the most efficient method for both intra-site and inter-site classifications to overcome the impact of spatial autocorrelation to avoid the potential to create a locally overfit model. Deep learning consistently outperformed random forest classification; both semi-supervised and supervised deep-learning models displayed the most promising results in creating a general taxa-classification model. This work has demonstrated the possibility of developing tree-classification models that can identify tree species from outside their training area and that semi-supervised deep learning may potentially utilize the untapped terabytes of unlabeled forest imagery. [ABSTRACT FROM AUTHOR]

Marine transportation accounts for approximately 90% of the total trade managed in international logistics and plays a vital role in many companies' supply chains. However, en-route factors like weather conditions or piracy incidents often delay scheduled arrivals at destination ports, leading to downstream inefficiencies. Due to the maritime industry's digital transformation, smart ports and vessels generate vast amounts of data, creating an opportunity to use the latest technologies, like machine and deep learning (ML/DL), to support terminals in their operations. This study proposes a data-driven solution for accurately predicting vessel arrival times using ML/DL techniques, including Deep Neural Networks, K-Nearest Neighbors, Decision Trees, Random Forest, and Extreme Gradient Boosting. This study collects real-world AIS data in the Eastern Mediterranean Sea from a network of public and private AIS base stations. The most relevant features are selected for training and evaluating the six ML/DL models. A comprehensive comparison is also performed against the estimated arrival time provided by shipping agents, a simple calculation-based approach, and four other ML/DL models proposed recently in the literature. The evaluation has revealed that Random Forest achieves the highest performance with an MAE of 99.9 min, closely followed by XGBoost, having an MAE of 105.0 min. [ABSTRACT FROM AUTHOR]

This study focuses on estimating the lead times of various processes in wind tower factories. Accurate estimation of these times allows for more efficient sequencing of activities, proper allocation of resources, and setting of realistic delivery dates, thus avoiding delays and bottlenecks in the production flow and improving process quality and efficiency. In addition, accurate estimation of these times contributes to a proper assessment of costs, overcoming the limitations of traditional techniques; this allows for the establishment of tighter quotations. The data used in this study were collected at wind tower manufacturing facilities in Spain and Brazil. Data preprocessing was conducted rigorously, encompassing cleaning, transformation, and feature selection processes. Following preprocessing, machine learning regression analysis was performed to estimate lead times. Nine algorithms were employed: decision trees, random forest, Ridge regression, Lasso regression, Elastic Net, support vector regression, gradient boosting, XGBoost, LightGBM, and multilayer perceptron. Additionally, the performance of two deep learning models, TabNet and NODE, designed specifically for tabular data, was evaluated. The results showed that gradient boosting-based algorithms were the most effective in predicting processing times and optimizing resource allocation. The system is designed to retrain models as new information becomes available. [ABSTRACT FROM AUTHOR]

Several machine learning (ML) techniques have demonstrated efficacy in precisely forecasting HIV risk and identifying the most eligible individuals for HIV testing in various countries. Nevertheless, there is a data gap on the utility of ML algorithms in strengthening HIV testing worldwide. This systematic review aimed to evaluate how effectively ML algorithms can enhance the efficiency and accuracy of HIV testing interventions and to identify key outcomes, successes, gaps, opportunities, and limitations in their implementation. This review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines. A comprehensive literature search was conducted via PubMed, Google Scholar, Web of Science, Science Direct, Scopus, and Gale OneFile databases. Out of the 845 identified articles, 51 studies were eligible. More than 75% of the articles included in this review were conducted in the Americas and various parts of Sub-Saharan Africa, and a few were from Europe, Asia, and Australia. The most common algorithms applied were logistic regression, deep learning, support vector machine, random forest, extreme gradient booster, decision tree, and the least absolute shrinkage selection operator model. The findings demonstrate that ML techniques exhibit higher accuracy in predicting HIV risk/testing compared to traditional approaches. Machine learning models enhance early prediction of HIV transmission, facilitate viable testing strategies to improve the efficiency of testing services, and optimize resource allocation, ultimately leading to improved HIV testing. This review points to the positive impact of ML in enhancing early prediction of HIV spread, optimizing HIV testing approaches, improving efficiency, and eventually enhancing the accuracy of HIV diagnosis. We strongly recommend the integration of ML into HIV testing programs for efficient and accurate HIV testing. [ABSTRACT FROM AUTHOR]

This project employs artificial intelligence, including machine learning and deep learning, to assess COVID-19 readmission risk in Malaysia. It offers tools to mitigate healthcare resource strain and enhance patient outcomes. This study outlines a methodology for classifying COVID-19 readmissions. It starts with dataset description and pre-processing, while the data balancing was computed through Random Oversampling, Borderline SMOTE, and Adaptive Synthetic Sampling. Nine machine learning and ten deep learning techniques are applied, with five-fold cross-validation for evaluation. Optuna is used for hyperparameter selection, while the consistency in training hyperparameters is maintained. Evaluation metrics encompass accuracy, AUC, and training/inference times. Results were based on stratified five-fold cross-validation and different data-balancing methods. Notably, CatBoost consistently excelled in accuracy and AUC across all tables. Using ROS, CatBoost achieved the highest accuracy (0.9882 ± 0.0020) with an AUC of 1.0000 ± 0.0000. CatBoost maintained its superiority in BSMOTE and ADASYN as well. Deep learning approaches performed well, with SAINT leading in ROS and TabNet leading in BSMOTE and ADASYN. Decision Tree ensembles like Random Forest and XGBoost consistently showed strong performance. [ABSTRACT FROM AUTHOR]

Airborne laser technology produces point clouds that can be used to build 3D models of buildings. However, the work is a laborious process that could benefit from automation. Artificial intelligence (AI) has been widely used in automating building segmentation as one of the initial stages in the 3D modeling process. The algorithms with a high success rate using point clouds for automatic semantic segmentation are random forest (RF) and PointNet++, with each algorithm having its own advantages and disadvantages. However, the training and testing data to develop and test the model usually share similar characteristics. Moreover, producing a good automation model requires a lot of training data, which may become an issue for users with a small amount of training data (limited data). The aim of this research is to test the performance of the RF and PointNet++ models in different regions with limited training and testing data. We found that the RF model developed from a small amount data, in different regions between the training and testing data, performs well compared to PointNet++, yielding an OA score of 73.01% for the RF model. Furthermore, several scenarios have been used in this research to explore the capabilities of RF in several cases. [ABSTRACT FROM AUTHOR]

Blockchain is a developing technology that promises advancements when it is applied to other fields. Applying blockchain to other systems requires a customized blockchain model to satisfy the requirements of different application fields. One important area is to integrate blockchain with smart spaces and the Internet of Things to process, manage, and store data. Actually, smart spaces and Internet of Things systems include various types of transactions in terms of sensitivity. The sensitivity can be considered as correctness sensitivity, time sensitivity, and specialization sensitivity. Correctness sensitivity means that the systems should accept precise or approximated data in some cases, whereas time sensitivity means that there are time bounds for each type of transaction, and specialization sensitivity applies when some transactions are processed only by specialized people. Therefore, this work introduces the smart partitioned blockchain model, where we use machine learning and deep learning models to classify transactions into different pools according to their sensitivity levels. Then, each pool is mapped to a specific part of the smart partitioned blockchain model. The parts can be permissioned or permissionless. The permissioned parts can have different sub-parts if needed. Consequently, the smart partitioned blockchain can be customized to meet application-field requirements. In the experimental results, we use bank and medical datasets with a predefined sensitivity threshold for classification accuracy in each system. The bank transactions are critical, whereas the classification of the medical dataset is speculative and less critical. The Random Forest model is used for bank-dataset classification, and its accuracy reaches 100%, whereas Sequential Deep Learning is used for the medical dataset, which reaches 91%. This means that all bank transactions are correctly mapped to the corresponding parts of the blockchain, whereas accuracy is lower for the medical dataset. However, acceptability is determined based on the predefined sensitivity threshold. [ABSTRACT FROM AUTHOR]

Intrusion Detection Systems (IDSs) play a crucial role in safeguarding network infrastructures from cyber threats and ensuring the integrity of highly sensitive data. Conventional IDS technologies, although successful in achieving high levels of accuracy, frequently encounter substantial model bias. This bias is primarily caused by imbalances in the data and the lack of relevance of certain features. This study aims to tackle these challenges by proposing an advanced machine learning (ML) based IDS that minimizes misclassification errors and corrects model bias. As a result, the predictive accuracy and generalizability of the IDS are significantly improved. The proposed system employs advanced feature selection techniques, such as Recursive Feature Elimination (RFE), sequential feature selection (SFS), and statistical feature selection, to refine the input feature set and minimize the impact of non-predictive attributes. In addition, this work incorporates data resampling methods such as Synthetic Minority Oversampling Technique and Edited Nearest Neighbor (SMOTE_ENN), Adaptive Synthetic Sampling (ADASYN), and Synthetic Minority Oversampling Technique–Tomek Links (SMOTE_Tomek) to address class imbalance and improve the accuracy of the model. The experimental results indicate that our proposed model, especially when utilizing the random forest (RF) algorithm, surpasses existing models regarding accuracy, precision, recall, and F Score across different data resampling methods. Using the ADASYN resampling method, the RF model achieves an accuracy of 99.9985% for botnet attacks and 99.9777% for Man-in-the-Middle (MITM) attacks, demonstrating the effectiveness of our approach in dealing with imbalanced data distributions. This research not only improves the abilities of IDS to identify botnet and MITM attacks but also provides a scalable and efficient solution that can be used in other areas where data imbalance is a recurring problem. This work has implications beyond IDS, offering valuable insights into using ML techniques in complex real-world scenarios. [ABSTRACT FROM AUTHOR]

Non-neural machine learning (ML) and deep learning (DL) are used to predict system failures in industrial maintenance. However, only a few studies have assessed the effect of varying the amount of past data used to make a prediction and the extension in the future of the forecast. This study evaluates the impact of the size of the reading window and of the prediction window on the performances of models trained to forecast failures in three datasets of (1) an industrial wrapping machine working in discrete sessions, (2) an industrial blood refrigerator working continuously, and (3) a nitrogen generator working continuously. A binary classification task assigns the positive label to the prediction window based on the probability of a failure to occur in such an interval. Six algorithms (logistic regression, random forest, support vector machine, LSTM, ConvLSTM, and Transformers) are compared on multivariate time series. The dimension of the prediction windows plays a crucial role and the results highlight the effectiveness of DL approaches in classifying data with diverse time-dependent patterns preceding a failure and the effectiveness of ML approaches in classifying similar and repetitive patterns preceding a failure. [ABSTRACT FROM AUTHOR]

The Yellow River Delta wetlands in China belong to the coastal wetland ecosystem, which is one of the youngest and most characteristic wetlands in the world. The Yellow River Delta wetlands are constantly changed by inland sediment and the influence of waves and storm surges, so the accurate classification of the coastal wetlands in the Yellow River Delta is of great significance for the rational utilization, development and protection of wetland resources. In this study, the Yellow River Delta sentinel-2 multispectral data were processed by super-resolution synthesis, and the feature bands were optimized. The optimal feature-band combination scheme was screened using the OIF algorithm. A deep learning model attention mechanism ResNet based on feature optimization with attention mechanism integration into the ResNet network is proposed. Compared with the classical machine learning model, the AM_ResNet model can effectively improve the classification accuracy of the wetlands in the Yellow River Delta. The overall accuracy was 94.61% with a Kappa of 0.93, and they were improved by about 6.99% and 0.1, respectively, compared with the best-performing Random Forest Classification in machine learning. The results show that the method can effectively improve the classification accuracy of the wetlands in the Yellow River Delta. [ABSTRACT FROM AUTHOR]

Gesture recognition has found widespread applications in various fields, such as virtual reality, medical diagnosis, and robot interaction. The existing mainstream gesture-recognition methods are primarily divided into two categories: inertial-sensor-based and camera-vision-based methods. However, optical detection still has limitations such as reflection and occlusion. In this paper, we investigate static and dynamic gesture-recognition methods based on miniature inertial sensors. Hand-gesture data are obtained through a data glove and preprocessed using Butterworth low-pass filtering and normalization algorithms. Magnetometer correction is performed using ellipsoidal fitting methods. An auxiliary segmentation algorithm is employed to segment the gesture data, and a gesture dataset is constructed. For static gesture recognition, we focus on four machine learning algorithms, namely support vector machine (SVM), backpropagation neural network (BP), decision tree (DT), and random forest (RF). We evaluate the model prediction performance through cross-validation comparison. For dynamic gesture recognition, we investigate the recognition of 10 dynamic gestures using Hidden Markov Models (HMM) and Attention-Biased Mechanisms for Bidirectional Long- and Short-Term Memory Neural Network Models (Attention-BiLSTM). We analyze the differences in accuracy for complex dynamic gesture recognition with different feature datasets and compare them with the prediction results of the traditional long- and short-term memory neural network model (LSTM). Experimental results demonstrate that the random forest algorithm achieves the highest recognition accuracy and shortest recognition time for static gestures. Moreover, the addition of the attention mechanism significantly improves the recognition accuracy of the LSTM model for dynamic gestures, with a prediction accuracy of 98.3%, based on the original six-axis dataset. [ABSTRACT FROM AUTHOR]

Petrographic prediction is crucial in identifying target areas and understanding reservoir lithology in oil and gas exploration. Traditional logging methods often rely on manual interpretation and experiential judgment, which can introduce subjectivity and constraints due to data quality and geological variability. To enhance the precision and efficacy of lithology prediction, this study employed a Savitzky–Golay filter with a symmetric window for anomaly data processing, coupled with a residual temporal convolutional network (ResTCN) model tasked with completing missing logging data segments. A comparative analysis against the support vector regression and random forest regression model revealed that the ResTCN achieves the smallest MAE, at 0.030, and the highest coefficient of determination, at 0.716, which are indicative of its proximity to the ground truth. These methodologies significantly enhance the quality of the training data. Subsequently, a Transformer–long short-term memory (T-LS) model was applied to identify and classify the lithology of unexplored wells. The input layer of the Transformer model follows an embedding-like principle for data preprocessing, while the encoding block encompasses multi-head attention, Add & Norm, and feedforward components, integrating the multi-head attention mechanism. The output layer interfaces with the LSTM layer through dropout. A performance evaluation of the T-LS model against established rocky prediction techniques such as logistic regression, k-nearest neighbor, and random forest demonstrated its superior identification and classification capabilities. Specifically, the T-LS model achieved a precision of 0.88 and a recall of 0.89 across nine distinct lithology features. A Shapley analysis of the T-LS model underscored the utility of amalgamating multiple logging data sources for lithology classification predictions. This advancement partially addresses the challenges associated with imprecise predictions and limited generalization abilities inherent in traditional machine learning and deep learning models applied to lithology identification, and it also helps to optimize oil and gas exploration and development strategies and improve the efficiency of resource extraction. [ABSTRACT FROM AUTHOR]

Airport service quality ratings found on social media such as Airline Quality and Google Maps offer invaluable insights for airport management to improve their quality of services. However, there is currently a lack of research analysing these reviews by airport services using sentimental analysis approaches. This research applies multiclass models based on Aspect-Based Sentimental Analysis to conduct a comprehensive analysis of travellers' reviews, in which the major airport services are tagged by positive, negative, and non-existent sentiments. Seven airport services commonly utilised in previous studies are also introduced. Subsequently, various Deep Learning architectures and Machine Learning classification algorithms are developed, tested, and compared using data collected from Twitter, Google Maps, and Airline Quality, encompassing travellers' feedback on airport service quality. The results show that the traditional Machine Learning algorithms such as the Random Forest algorithm outperform Deep Learning models in the multiclass prediction of airport service quality using travellers' feedback. The findings of this study offer concrete justifications for utilising multiclass Machine Learning models to understand the travellers' sentiments and therefore identify airport services required for improvement. [ABSTRACT FROM AUTHOR]

Detecting road cracks is essential for inspecting and assessing the integrity of concrete pavement structures. Traditional image-based methods often require complex preprocessing to extract crack features, making them challenging when dealing with noisy concrete surfaces in diverse real-world scenarios, such as autonomous vehicle road detection. This study introduces an image-based crack detection approach that combines a Random Forest machine learning classifier with a deep convolutional neural network (CNN) to address these challenges. Three state-of-the-art models, namely MobileNet, InceptionV3, and Xception, were employed and trained using a dataset of 30,000 images to build an effective CNN. A systematic comparison of validation accuracy across various base learning rates identified a base learning rate of 0.001 as optimal, achieving a maximum validation accuracy of 99.97%. This optimal learning rate was then applied in the subsequent testing phase. The robustness and flexibility of the trained models were evaluated using 6,000 test photos, each with a resolution of 224 × 224 pixels, which were not part of the training or validation sets. The outstanding results, boasting a remarkable 99.95% accuracy, 99.95% precision, 99.94% recall, and a matching 99.94% F1 Score, unequivocally affirm the efficacy of the proposed technique in precisely identifying road fractures in photographs taken on real concrete surfaces. [ABSTRACT FROM AUTHOR]

Developing a fast and non-destructive methodology to identify the storage years of Coix seed is important in safeguarding consumer well-being. This study employed the utilization of hyperspectral imaging (HSI) in conjunction with conventional machine learning techniques such as support vector machines (SVM), k-nearest neighbors (KNN), random forest (RF), extreme gradient boosting (XGBoost), as well as the deep learning method of residual neural network (ResNet), to establish identification models for Coix seed samples from different storage years. Under the fusion-based modeling approach, the model's classification accuracy surpasses that of visible to near infrared (VNIR) and short-wave infrared (SWIR) spectral modeling individually. The classification accuracy of the ResNet model and SVM exceeds that of other conventional machine learning models (KNN, RF, and XGBoost). Redundant variables were further diminished through competitive adaptive reweighted sampling feature wavelength screening, which had less impact on the model's accuracy. Upon validating the model's performance using an external validation set, the ResNet model yielded more satisfactory outcomes, exhibiting recognition accuracy exceeding 85%. In conclusion, the comprehensive results demonstrate that the integration of deep learning with HSI techniques effectively distinguishes Coix seed samples from different storage years. [ABSTRACT FROM AUTHOR]

We present an evaluation of different deep learning and machine learning approaches for tree health classification in the Black Forest, the Harz Mountains, and the Göttinger Forest on a unique, highly accurate tree-level dataset. The multispectral UAV data were collected from eight forest plots with diverse tree species, mostly conifers. As ground truth data (GTD), nearly 1500 tree polygons with related attribute information on the health status of the trees were used. This data were collected during extensive fieldwork using a mobile application and subsequent individual tree segmentation. Extensive preprocessing included normalization, NDVI calculations, data augmentation to deal with the underrepresented classes, and splitting the data into training, validation, and test sets. We conducted several experiments using a classical machine learning approach (random forests), as well as different convolutional neural networks (CNNs)—ResNet50, ResNet101, VGG16, and Inception-v3—on different datasets and classes to evaluate the potential of these algorithms for tree health classification. Our first experiment was a binary classifier of healthy and damaged trees, which did not consider the degree of damage or tree species. The best results of a 0.99 test accuracy and an F1 score of 0.99 were obtained with ResNet50 on four band composites using the red, green, blue, and infrared bands (RGBI images), while VGG16 had the worst performance, with an F1 score of only 0.78. In a second experiment, we also distinguished between coniferous and deciduous trees. The F1 scores ranged from 0.62 to 0.99, with the highest results obtained using ResNet101 on derived vegetation indices using the red edge band of the camera (NDVIre images). Finally, in a third experiment, we aimed at evaluating the degree of damage: healthy, slightly damaged, and medium or heavily damaged trees. Again, ResNet101 had the best performance, this time on RGBI images with a test accuracy of 0.98 and an average F1 score of 0.97. These results highlight the potential of CNNs to handle high-resolution multispectral UAV data for the early detection of damaged trees when good training data are available. [ABSTRACT FROM AUTHOR]

Objective: This research addresses the challenges of maintaining proper yoga postures, an issue that has been exacerbated by the COVID-19 pandemic and the subsequent shift to virtual platforms for yoga instruction. This research aims to develop a mechanism for detecting correct yoga poses and providing real-time feedback through the application of computer vision and machine learning (ML) techniques. Methods and Procedures: This study utilized computer vision-based pose estimation methods to extract features and calculate yoga pose angles. A variety of models, including extremely randomized trees, logistic regression, random forest, gradient boosting, extreme gradient boosting, and deep neural networks, were trained and tested to classify yoga poses. Our study employed the Yoga-82 dataset, consisting of many yoga pose images downloaded from the web. Results: The results of this study show that the extremely randomized trees model outperformed the other models, achieving the highest prediction accuracy of 91% on the test dataset and 92% in a fivefold cross-validation experiment. Other models like random forest, gradient boosting, extreme gradient boosting, and deep neural networks achieved accuracies of 90%, 89%, 90%, and 85%, respectively, while logistic regression underperformed, having the lowest accuracy. Conclusion: This research concludes that the extremely randomized trees model presents superior predictive power for yoga pose recognition. This suggests a valuable avenue for future exploration in this domain. Moreover, the approach has significant potential for implementation on low-powered smartphones with minimal latency, thereby enabling real-time feedback for users practicing yoga at home. [ABSTRACT FROM AUTHOR]

The occurrence of rockburst can cause significant disasters in underground rock engineering. It is crucial to predict and prevent rockburst in deep tunnels and mines. In this paper, the deficiencies of ensemble learning algorithms in rockburst prediction were investigated. Aiming at these shortages, a novel machine learning model, deep forest, was proposed to predict rockburst risk. The deep forest combines the characteristics of deep learning and ensemble models, which can solve complex problems. To develop the deep forest model for rockburst prediction, 329 real rockburst cases were collected to build a comprehensive database for intelligent analysis. Bayesian optimization was proposed to tune the hyperparameters of the deep forest. As a result, the deep forest model achieved 100% training accuracy and 92.4% testing accuracy, and it has more outstanding capability to forecast rockburst disasters compared to other widely used models (i.e., random forest, boosting tree models, neural network, support vector machine, etc.). The results of sensitivity analysis revealed the impact of variables on rockburst levels and the applicability of deep forest with a few input parameters. Eventually, real cases of rockburst in two gold mines, China, were used for validation purposes while the needed data sets were prepared by field observations and laboratory tests. The promoting results of the developed model during the validation phase confirm that it can be used with a high level of accuracy by practicing engineers for predicting rockburst occurrences. [ABSTRACT FROM AUTHOR]

This paper presents a computer-vision-based methodology for automatic image-based classification of 2042 training images and 284 unseen (test) images divided into 68 categories of gemstones. A series of feature extraction techniques (33 including colour histograms in the RGB, HSV and CIELAB space, local binary pattern, Haralick texture and grey-level co-occurrence matrix properties) were used in combination with different machine-learning algorithms (Logistic Regression, Linear Discriminant Analysis, K-Nearest Neighbour, Decision Tree, Random Forest, Naive Bayes and Support Vector Machine). Deep-learning classification with ResNet-18 and ResNet-50 was also investigated. The optimal combination was provided by a Random Forest algorithm with the RGB eight-bin colour histogram and local binary pattern features, with an accuracy of 69.4% on unseen images; the algorithms required 0.0165 s to process the 284 test images. These results were compared against three expert gemmologists with at least 5 years of experience in gemstone identification, who obtained accuracies between 42.6% and 66.9% and took 42–175 min to classify the test images. As expected, the human experts took much longer than the computer vision algorithms, which in addition provided, albeit marginal, higher accuracy. Although these experiments included a relatively low number of images, the superiority of computer vision over humans is in line with what has been reported in other areas of study, and it is encouraging to further explore the application in gemmology and related areas. [ABSTRACT FROM AUTHOR]

The exploitation of higher millimeter wave (MmWave) is promising for wireless communication systems. The goals of machine learning (ML) and its subcategories of deep learning beyond 5G (B5G) is to learn from the data and make a prediction or a decision other than relying on the classical procedures to enhance the wireless design. The new wireless generation should be proactive and predictive to avoid the previous drawbacks in the existing wireless generations to meet the 5G target services pillars. One of the aspects of Ultra-Reliable Low Latency Communications (URLLC) is moving the data processing tasks to the cellular base stations. With the rapid usage of wireless communications devices, base stations are required to execute and make decisions to ensure communication reliability. In this paper, an efficient new methodology using ML is applied to assist base stations in predicting the frequency bands and the path loss based on a data-driven approach. The ML algorithms that are used and compared are Multilelayers Perceptrons (MLP) as a neural networks branch and Random Forests. Systems that consume different bands such as base stations in telecommunications with uplink and downlink transmissions and other internet of things (IoT) devices need an urgent response between devices to alter bands to maintain the requirements of the new radios (NR). Thus, ML techniques are needed to learn and assist a base station to fluctuate between different bands based on a data-driven system. Then, to testify the proposed idea, we compare the analysis with other deep learning methods. Furthermore, to validate the proposed models, we applied these techniques to different case studies to ensure the success of the proposed works. To enhance the accuracy of supervised data learning, we modified the random forests by combining an unsupervised algorithm to the learning process. Eventually, the superiority of ML towards wireless communication demonstrated great accuracy at 90.24%. [ABSTRACT FROM AUTHOR]

Existing accelerometer-based human activity recognition (HAR) benchmark datasets that were recorded during free living suffer from non-fixed sensor placement, the usage of only one sensor, and unreliable annotations. We make two contributions in this work. First, we present the publicly available Human Activity Recognition Trondheim dataset (HARTH). Twenty-two participants were recorded for 90 to 120 min during their regular working hours using two three-axial accelerometers, attached to the thigh and lower back, and a chest-mounted camera. Experts annotated the data independently using the camera's video signal and achieved high inter-rater agreement (Fleiss' Kappa = 0.96 ). They labeled twelve activities. The second contribution of this paper is the training of seven different baseline machine learning models for HAR on our dataset. We used a support vector machine, k-nearest neighbor, random forest, extreme gradient boost, convolutional neural network, bidirectional long short-term memory, and convolutional neural network with multi-resolution blocks. The support vector machine achieved the best results with an F1-score of 0.81 (standard deviation: ± 0.18 ), recall of 0.85 ± 0.13 , and precision of 0.79 ± 0.22 in a leave-one-subject-out cross-validation. Our highly professional recordings and annotations provide a promising benchmark dataset for researchers to develop innovative machine learning approaches for precise HAR in free living. [ABSTRACT FROM AUTHOR]

Efficient land management and farming practices are critical to maintaining agricultural production, especially in Europe with limited arable land. It is very time consuming to rely on a manual field inspection of cultivated land to archive farm crops. But with the help of satellite monitoring data on the earth's surface, it is a new vision to classify farmland based on deep learning. This article has studied the Sentinel 2 (S2) data, which are top-of-atmosphere (TOA) reflectance values at the processing level-1C (L1C) observed from some areas of Germany and France. Aiming at the problem that the interference of atmosphere and cloud coverage weakens the recognition accuracy of subsequent algorithms, a method of combining feature expansion and feature importance analysis is proposed to optimize the raw S2 data. Specifically, the new 13 spectral features are expanded based on the linear and nonlinear combination of the raw 13 spectral bands of S2. The random forest (RF) algorithm is used to score the importance of features, and the important features of each time series are selected to form a new dataset. Then, an end-to-end deep learning model has been used for training. The structure of the model is a two-layer unidirectional recurrent neural network with long short-term memory (LSTM) as the backbone. And two linear layers as the output, which form two decision-making heads, respectively, representing output classification probability and the stop decision. The results show that adding features and selecting features is beneficial for the model to improve classification accuracy and predict the classification without all of the input data. This end-to-end classification pattern with early prediction would support intelligent monitoring of farm crops with a great advantage to the implementation of various agricultural policies. [ABSTRACT FROM AUTHOR]