Search

Skin cancer has a significant impact on the lives of many individuals annually and is recognized as the most prevalent type of cancer. In the United States, an estimated annual incidence of approximately 3.5 million people receiving a diagnosis of skin cancer underscores its widespread prevalence. Furthermore, the prognosis for individuals afflicted with advancing stages of skin cancer experiences a substantial decline in survival rates. This paper is dedicated to aiding healthcare experts in distinguishing between benign and malignant skin cancer cases by employing a range of machine learning and deep learning techniques and different feature extractors and feature selectors to enhance the evaluation metrics. In this paper, different transfer learning models are employed as feature extractors, and to enhance the evaluation metrics, a feature selection layer is designed, which includes diverse techniques such as Univariate, Mutual Information, ANOVA, PCA, XGB, Lasso, Random Forest, and Variance. Among transfer models, DenseNet-201 was selected as the primary feature extractor to identify features from data. Subsequently, the Lasso method was applied for feature selection, utilizing diverse machine learning approaches such as MLP, XGB, RF, and NB. To optimize accuracy and precision, ensemble methods were employed to identify and enhance the best-performing models. The study provides accuracy and sensitivity rates of 87.72% and 92.15%, respectively. [ABSTRACT FROM AUTHOR]

Cardiovascular disease (CVD) is a leading cause of death globally; therefore, early detection of CVD is crucial. Many intelligent technologies, including deep learning and machine learning (ML), are being integrated into healthcare systems for disease prediction. This paper uses a voting ensemble ML with chi-square feature selection to detect CVD early. Our approach involved applying multiple ML classifiers, including naïve Bayes, random forest, logistic regression (LR), and k-nearest neighbor. These classifiers were evaluated through metrics including accuracy, specificity, sensitivity, F1-score, confusion matrix, and area under the curve (AUC). We created an ensemble model by combining predictions from the different ML classifiers through a voting mechanism, whose performance was then measured against individual classifiers. Furthermore, we applied chi-square feature selection method to the 303 records across 13 clinical features in the Cleveland cardiac disease dataset to identify the 5 most important features. This approach improved the overall accuracy of our ensemble model and reduced the computational load considerably by more than 50%. Demonstrating superior effectiveness, our voting ensemble model achieved a remarkable accuracy of 92.11%, representing an average improvement of 2.95% over the single highest classifier (LR). These results indicate the ensemble method as a viable and practical approach to improve the accuracy of CVD prediction. [ABSTRACT FROM AUTHOR]

Particulate matter (PM) in the air can cause various health problems and diseases in humans. In particular, the smaller size of PM 2.5 enable them to penetrate deep into the lungs, causing severe health impacts. Exposure to PM 2.5 can result in respiratory, cardiovascular, and allergic diseases, and prolonged exposure has also been linked to an increased risk of cancer, including lung cancer. Therefore, forecasting the PM 2.5 concentration in the surrounding is crucial for preventing these adverse health effects. This paper proposes a method for forecasting the PM 2.5 concentration after 1 h using bidirectional long short-term memory (Bi-LSTM). The proposed method involves selecting input variables based on the feature importance calculated by random forest, classifying the data to assign weight variables to reduce bias, and forecasting the PM 2.5 concentration using Bi-LSTM. To compare the performance of the proposed method, two case studies were conducted. First, a comparison of forecasting performance according to preprocessing. Second, forecasting performance between deep learning (long short-term memory, gated recurrent unit, and Bi-LSTM) and conventional machine learning models (multi-layer perceptron, support vector machine, decision tree, and random forest). In case study 1, The proposed method shows that the performance indices (RMSE: 3.98%p, MAE: 5.87%p, RRMSE: 3.96%p, and R 2 :0.72%p) are improved because weights are given according to the input variables before the forecasting is performed. In case study 2, we show that Bi-LSTM, which considers both directions (forward and backward), can effectively forecast when compared to conventional models (RMSE: 2.70, MAE: 0.84, RRMSE: 1.97, R 2 : 0.16). Therefore, it is shown that the proposed method can effectively forecast PM 2.5 even if the data in the high-concentration section is insufficient. [ABSTRACT FROM AUTHOR]

Anomaly-based network intrusion detection systems are highly significant in detecting network attacks. Robust machine learning and deep learning models for identifying network intrusion and attack types are proposed in this paper. Proposed models have experimented with the UNSW-NB15 dataset of 49 features for nine different attack samples. Decision Tree classifier produced the best accuracy of 99.05% compared to ensemble models - Random forest(98.96%), Adaboost(97.87%), and XGBoost(98.08%). K-Nearest Neighbour classifier trained for various values of K and best performance obtained for K=7 with the accuracy of 95.58%. A Deep Learning model with two dense layers with ReLU activation and a third dense layer with a Sigmoid activation function is designed for binary classification and produced good accuracy of 98.44% with ADAM optimizer, 80:20 Train-Test Split Ratio. Network attack exploits are detected with an accuracy 95% by XGBoost, Fuzzers attack with 90% accuracy by Random Forest, Generic attacks with 99% accuracy by Random Forest, and Reconnaissance attacks with 79% by Decision Trees. All features are relevant and strong in network attack detection, which eliminates the requirement of feature selection. [ABSTRACT FROM AUTHOR]

Patient-centered communication (PCC) quality is critical to increasing the quality of patient-centered care. Based on the nationally representative data of the Health Information National Trends Survey (HINTS) 2019–2020 (N = 4593), this study combined four machine learning methods, namely, Generalized Linear Models (GLM), Random Forests (Random Forests), Deep Neural Networks (Deep Learning), and Gradient Boosting Machines (GBM), to identify important PCC predictors through variable importance metrics. Fifteen variables were identified as important predictors, involving multiple dimensions, such as individual sociodemographic characteristics, health-related factors, and individual living habits. Among them, four novel potential associated variables are included, an individual's level of verbal expression, exercise habits, etc., which significantly impacted respondents' perceived PCC quality. This study revealed the value of combining feature selection with machine learning approaches to identify broad variables that could enhance PCC prediction and clinical decision-making, influence future PCC prediction research, and improve patient-centered care. In the future, other easy-to-interpret models can be combined to conduct further research on the impact direction and mechanism of important predictors on PCC. [ABSTRACT FROM AUTHOR]

This study focuses on estimating the lead times of various processes in wind tower factories. Accurate estimation of these times allows for more efficient sequencing of activities, proper allocation of resources, and setting of realistic delivery dates, thus avoiding delays and bottlenecks in the production flow and improving process quality and efficiency. In addition, accurate estimation of these times contributes to a proper assessment of costs, overcoming the limitations of traditional techniques; this allows for the establishment of tighter quotations. The data used in this study were collected at wind tower manufacturing facilities in Spain and Brazil. Data preprocessing was conducted rigorously, encompassing cleaning, transformation, and feature selection processes. Following preprocessing, machine learning regression analysis was performed to estimate lead times. Nine algorithms were employed: decision trees, random forest, Ridge regression, Lasso regression, Elastic Net, support vector regression, gradient boosting, XGBoost, LightGBM, and multilayer perceptron. Additionally, the performance of two deep learning models, TabNet and NODE, designed specifically for tabular data, was evaluated. The results showed that gradient boosting-based algorithms were the most effective in predicting processing times and optimizing resource allocation. The system is designed to retrain models as new information becomes available. [ABSTRACT FROM AUTHOR]

Intrusion Detection Systems (IDSs) play a crucial role in safeguarding network infrastructures from cyber threats and ensuring the integrity of highly sensitive data. Conventional IDS technologies, although successful in achieving high levels of accuracy, frequently encounter substantial model bias. This bias is primarily caused by imbalances in the data and the lack of relevance of certain features. This study aims to tackle these challenges by proposing an advanced machine learning (ML) based IDS that minimizes misclassification errors and corrects model bias. As a result, the predictive accuracy and generalizability of the IDS are significantly improved. The proposed system employs advanced feature selection techniques, such as Recursive Feature Elimination (RFE), sequential feature selection (SFS), and statistical feature selection, to refine the input feature set and minimize the impact of non-predictive attributes. In addition, this work incorporates data resampling methods such as Synthetic Minority Oversampling Technique and Edited Nearest Neighbor (SMOTE_ENN), Adaptive Synthetic Sampling (ADASYN), and Synthetic Minority Oversampling Technique–Tomek Links (SMOTE_Tomek) to address class imbalance and improve the accuracy of the model. The experimental results indicate that our proposed model, especially when utilizing the random forest (RF) algorithm, surpasses existing models regarding accuracy, precision, recall, and F Score across different data resampling methods. Using the ADASYN resampling method, the RF model achieves an accuracy of 99.9985% for botnet attacks and 99.9777% for Man-in-the-Middle (MITM) attacks, demonstrating the effectiveness of our approach in dealing with imbalanced data distributions. This research not only improves the abilities of IDS to identify botnet and MITM attacks but also provides a scalable and efficient solution that can be used in other areas where data imbalance is a recurring problem. This work has implications beyond IDS, offering valuable insights into using ML techniques in complex real-world scenarios. [ABSTRACT FROM AUTHOR]

Parkinson's disease (PD) harms the human brain's nervous system and can affect the patient's life. However, the diagnosis of PD diagnosis in the first stages can lead to early treatment and save costs. In this paper, a cloud-based machine learning diagnosing intelligent system is proposed for the PD with respect to patient voice. The proposed system is composed of two stages. In the first stage, two machine learning approaches, Random-Forest (RF) and Long-Short-Term-Memory (LSTM) are applied to generate a model that can be used for early treatment of PD. In this stage, a feature selection method is used to choose the minimum subset of the best features, which can be utilized later to generate the classification model. In the second stage, the best diagnosis model is deployed in cloud computing. In this stage, an Android application is also designed to provide the interface to the diagnosis model. The performance evaluation of the diagnosis model is conducted based on the F-score accuracy measurement. The result shows that the LTSM model has superior accuracy with 95% of the F-score compared with the RF model. Therefore, the LSTM model is selected for implementing a cloudbased PD diagnosing application using Python and Java. [ABSTRACT FROM AUTHOR]

Approximately 65% of the worldwide adult population has experienced stress, affecting their daily routine at least once in the past year. Stress becomes harmful when it occurs for too long or is continuous (i.e., chronic), interfering with our performance, attention, and concentration. Chronic high stress contributes to major health issues such as heart disease, high blood pressure, diabetes, depression, and anxiety. Several researchers have focused on detecting stress through combining many features with machine/deep learning models. Despite these efforts, our community has not agreed on the number of features to identify stress conditions using wearable devices. In addition, most of the reported studies have been focused on person-specific training and testing. Thanks to our community's broad acceptance of wearable wristband devices, this work investigates a global stress detection model combining eight HRV features with a random forest (RF) algorithm. Whereas the model's performance is evaluated for each individual, the training of the RF model contains instances of all subjects (i.e., global training). We have validated the proposed global stress model using two open-access databases (the WESAD and SWELL databases) and their combination. The eight HRV features with the highest classifying power are selected using the minimum redundancy maximum relevance (mRMR) method, reducing the training time of the global stress platform. The proposed global stress monitoring model identifies person-specific stress events with an accuracy higher than 99% after a global training framework. Future work should be focused on testing this global stress monitoring framework in real-world applications. [ABSTRACT FROM AUTHOR]

Background: the credit scoring model is an effective tool for banks and other financial institutions to distinguish potential default borrowers. The credit scoring model represented by machine learning methods such as deep learning performs well in terms of the accuracy of default discrimination, but the model itself also has many shortcomings such as many hyperparameters and large dependence on big data. There is still a lot of room to improve its interpretability and robustness. Methods: the deep forest or multi-Grained Cascade Forest (gcForest) is a decision tree depth model based on the random forest algorithm. Using multidimensional scanning and cascading processing, gcForest can effectively identify and process high-dimensional feature information. At the same time, gcForest has fewer hyperparameters and has strong robustness. So, this paper constructs a two-stage hybrid default discrimination model based on multiple feature selection methods and gcForest algorithm, and at the same time, it optimizes the parameters for the lowest type II error as the first principle, and the highest AUC and accuracy as the second and third principles. GcForest can not only reflect the advantages of traditional statistical models in terms of interpretability and robustness but also take into account the advantages of deep learning models in terms of accuracy. Results: the validity of the hybrid default discrimination model is verified by three real open credit data sets of Australian, Japanese, and German in the UCI database. Conclusions: the performance of the gcForest is better than the current popular single classifiers such as ANN, and the common ensemble classifiers such as LightGBM, and CNNs in type II error, AUC, and accuracy. Besides, in comparison with other similar research results, the robustness and effectiveness of this model are further verified. [ABSTRACT FROM AUTHOR]

Cryptojacking or illegal mining is a form of malware that hides in the victim's computer and takes the computational resources to extract cryptocurrencies in favor of the attacker. It generates significant computational consumption, reducing the computational efficiency of the victim's computer. This attack has increased due to the rise of cryptocurrencies and their profitability and its difficult detection by the user. The identification and blocking of this type of malware have become an aspect of research related to cryptocurrencies and blockchain technology; in the literature, some machine learning and deep learning techniques are presented, but they are still susceptible to improvement. In this work, we explore multiple Machine Learning classification models for detecting cryptojacking on websites, such as Logistic Regression, Decision Tree, Random Forest, Gradient Boosting Classifier, k-Nearest Neighbor, and XGBoost. To this end, we make use of a dataset, composed of network and host features' samples, to which we apply various feature selection methods such as those based on statistical methods, e.g., Test Anova, and other methods as Wrappers, not only to reduce the complexity of the built models but also to discover the features with the greatest predictive power. Our results suggest that simple models such as Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, and k-Nearest Neighbor models, can achieve success rate similar to or greater than that of advanced algorithms such as XGBoost and even those of other works based on Deep Learning. [ABSTRACT FROM AUTHOR]

Genomic Prediction (GP) is a powerful approach for inferring complex phenotypes from genetic markers. GP is critical for improving grain yield, particularly for staple crops such as wheat and rice, which are crucial to feeding the world. While machine learning (ML) models have recently started to be applied in GP, it is often unclear what are the best algorithms and how their results are affected by the feature selection (FS) methods. Here, we compared ML and deep learning (DL) algorithms with classical Bayesian approaches, across a range of different FS methods, for their performance in predicting wheat grain yield (in three datasets). Model performance was generally more affected by the prediction algorithm than the FS method. Among all models, the best performance was obtained for tree-based ML methods (random forests and gradient boosting) and for classical Bayesian methods. However, the latter was prone to fitting problems. This issue was also observed for models developed with features selected by BayesA, the only Bayesian FS method used here. Nonetheless, the three other FS methods led to models with no fitting problem but similar performance. Thus, our results indicate that the choice of prediction algorithm is more important than the choice of FS method for developing highly predictive models. Moreover, we concluded that random forests and gradient boosting algorithms generate highly predictive and robust wheat grain yield GP models. [ABSTRACT FROM AUTHOR]

Recent growth in crop genomic and trait data have opened opportunities for the application of novel approaches to accelerate crop improvement. Machine learning and deep learning are at the forefront of prediction-based data analysis. However, few approaches for genotype to phenotype prediction compare machine learning with deep learning and further interpret the models that support the predictions. This study uses genome wide molecular markers and traits across 1110 soybean individuals to develop accurate prediction models. For 13/14 sets of predictions, XGBoost or random forest outperformed deep learning models in prediction performance. Top ranked SNPs by F-score were identified from XGBoost, and with further investigation found overlap with significantly associated loci identified from GWAS and previous literature. Feature importance rankings were used to reduce marker input by up to 90%, and subsequent models maintained or improved their prediction performance. These findings support interpretable machine learning as an approach for genomic based prediction of traits in soybean and other crops. [ABSTRACT FROM AUTHOR]

Accurate forecasting of future meteorological elements is critical and has profoundly affected human life in many aspects from rainstorm warning to flight safety. The conventional numerical weather prediction (NWP) sometimes leads to unsatisfactory performance due to inappropriate initial state settings. In this paper, a short-term weather forecasting model based on wavelet packet denoising and Catboost is proposed, which takes advantage of the fusion information combining the historical observation data with the prior knowledge from NWP. The feature selection and spatiotemporal feather addition are also explored to further improve performance. The proposed method is evaluated on the datasets provided by Beijing weather stations. Experimental results demonstrate that compared with many deep-learning or machine-learning methods such as LSTM, Seq2Seq, and random forest, the proposed Catboost model incorporated with wavelet packet denoising can achieve shorter convergence time and higher prediction accuracy. [ABSTRACT FROM AUTHOR]

Seismic characterisation of deep carbonate reservoirs is of considerable interest for reservoir distribution prediction, reservoir quality evaluation and reservoir structure delineation. However, it is challenging to use the traditional methodology to predict a deep-buried carbonate reservoir because of the highly nonlinear mapping relationship between heterogeneous reservoir features and seismic responses. We propose a machine-learning-based method (random forest) with physical constraints to enhance deep carbonate reservoir prediction performance from multi-seismic attributes. We demonstrate the effectiveness of this method on a real data application in the deep carbonate reservoir of Tarim Basin, Western China. We first perform feature selection on multi-seismic attributes, then four kinds of physical constraint (continuity, boundary, spatial and category constraint) transferred from domain knowledge are imposed on the process of model building. Using the physical constraints, the F1 score of reservoir quality and reservoir type can be significantly improved and the combination of the effective physical constraints gives the best prediction of performance. We also apply the proposed strategy on 2D seismic data to predict the spatial distribution of reservoir quality and type. The seismic prediction results provide a reasonable description of the strong heterogeneity of the reservoir, offering insights into sweet spot detection and reservoir development. [ABSTRACT FROM AUTHOR]

Breast cancer resistance protein (BCRP/ABCG2), an ATP-binding cassette (ABC) efflux transporter, plays a critical role in multi-drug resistance (MDR) to anti-cancer drugs and drug–drug interactions. The prediction of BCRP inhibition can facilitate evaluating potential drug resistance and drug–drug interactions in early stage of drug discovery. Here we reported a structurally diverse dataset consisting of 1098 BCRP inhibitors and 1701 non-inhibitors. Analysis of various physicochemical properties illustrates that BCRP inhibitors are more hydrophobic and aromatic than non-inhibitors. We then developed a series of quantitative structure–activity relationship (QSAR) models to discriminate between BCRP inhibitors and non-inhibitors. The optimal feature subset was determined by a wrapper feature selection method named rfSA (simulated annealing algorithm coupled with random forest), and the classification models were established by using seven machine learning approaches based on the optimal feature subset, including a deep learning method, two ensemble learning methods, and four classical machine learning methods. The statistical results demonstrated that three methods, including support vector machine (SVM), deep neural networks (DNN) and extreme gradient boosting (XGBoost), outperformed the others, and the SVM classifier yielded the best predictions (MCC = 0.812 and AUC = 0.958 for the test set). Then, a perturbation-based model-agnostic method was used to interpret our models and analyze the representative features for different models. The application domain analysis demonstrated the prediction reliability of our models. Moreover, the important structural fragments related to BCRP inhibition were identified by the information gain (IG) method along with the frequency analysis. In conclusion, we believe that the classification models developed in this study can be regarded as simple and accurate tools to distinguish BCRP inhibitors from non-inhibitors in drug design and discovery pipelines. [ABSTRACT FROM AUTHOR]