Search

Recent advancements in information and communication technologies have led to a proliferation of online systems and services. To ensure these systems' trustworthiness and prevent cybersecurity threats, Intrusion Detection Systems (IDS) are essential. Therefore, developing advanced and intelligent IDS models has become crucial. However, most existing IDS models rely on traditional machine learning algorithms with shallow learning behaviours, resulting in less efficient feature selection and classification performance for new attacks. Another issue is that these approaches are either network-based or host-based, often leading to the detection module missing many known attacks. Additionally, they struggle to handle the massive amounts of network traffic data flexible and scalable due to high model complexity. To address these challenges, an efficient hybrid IDS model is introduced, utilizing a MapReduce-based Black Widow Optimized Convolutional-Long Short-Term Memory (BWO-CONV-LSTM) network. The first stage of this IDS model involves feature selection using the Artificial Bee Colony (ABC) algorithm. The second stage employs a hybrid deep learning classifier model of BWO-CONV-LSTM on a MapReduce framework for intrusion detection from system traffic data. The proposed BWO-CONV-LSTM network combines Convolutional and LSTM neural networks, with hyper-parameters optimized by BWO to achieve the ideal architecture. The BWO-CONV-LSTM-based IDS model performance evaluations were conducted on the NSL-KDD, ISCX-IDS, UNSWNB15, and CSE-CIC-IDS2018 datasets. The results show that the proposed model achieves high intrusion detection performance, with accuracy rates of 98.67%, 97.003%, 98.667%, and 98.25% for the NSL-KDD, ISCX-IDS, UNSWNB15, and CSE-CIC-IDS2018 datasets, respectively. It also demonstrates fewer false values, reduced computation time, and improved classification coefficients. [ABSTRACT FROM AUTHOR]

The coronavirus occurred in Wuhan (China) first and it was declared a global pandemic. To detect coronavirus X-ray images can be used. Convolutional neural networks (CNNs) are used commonly to detect illness from images. There can be lots of different alternative deep CNN models or architectures. To find the best architecture, hyper-parameter optimization can be used. In this study, the problem is modeled as a multi-objective optimization (MOO) problem. Objective functions are multi-class cross entropy, error ratio, and complexity of the CNN network. For the best solutions to the objective functions, multi-objective hyper-parameter optimization is made by NSGA-III, NSGA-II, R-NSGA-II, SMS-EMOA, MOEA/D, and proposed Swarm Genetic Algorithms (SGA). SGA is a swarm-based algorithm with a cross-over process. All six algorithms are run and give Pareto optimal solution sets. When the figures obtained from the algorithms are analyzed and algorithm hypervolume values are compared, SGA outperforms the NSGA-III, NSGA-II, R-NSGA-II, SMS-EMOA, and MOEA/D algorithms. It can be concluded that SGA is better than others for multi-objective hyper-parameter optimization algorithms for COVID-19 detection from X-ray images. Also, a sensitivity analysis has been made to understand the effect of the number of the parameters of CNN on model success. [ABSTRACT FROM AUTHOR]

Abstract This study investigates the application of various regression models for predicting drug solubility in polymer and API-polymer interactions in complex datasets. Four models—Gaussian Process Regression (GPR), Support Vector Regression (SVR), Bayesian Ridge Regression (BRR), and Kernel Ridge Regression (KRR)—are evaluated. Preprocessing the dataset using the Z-score approach helped to detect outliers, further improving the accuracy and dependability of the analysis. Also, Fireworks Algorithm (FWA) is employed for hyper-parameter tuning in this work. The GPR model demonstrated superior performance, achieving the lowest MSE and MAE for both drug solubility and gamma predictions, with R2 scores of 0.9980 and 0.9950 for training and test data, respectively. The results of this study show the robustness of GPR in generating reliable and precise forecasts, thus providing a strong method for intricate regression tasks in pharmaceutical and other scientific fields. In addition, the Fireworks Algorithm (FWA) is presented as an optimization method, demonstrating its potential in improving the model’s predictive abilities by effectively exploring and exploiting the search space. The results emphasize the significance of choosing suitable regression models and optimization techniques to attain dependable and superior predictive analytics.

The constraints of spatiotemporal heterogeneity and spatial resolution constitute two crucial challenges in the establishment of remote sensing inversion models. Spatiotemporal heterogeneity gives rise to an inadequate generalization capacity of remote sensing models, demanding extensive manual parameter adjustment for each model construction. This not only escalates the task's work intensity but also leads to unstable performance. The limited spatial resolution of remote sensing images leads to suboptimal inversion accuracy for sampling points influenced by mixed pixel effects. To tackle these problems, we take the case of non-photosensitive water quality parameter inversion in the narrow rivers of Longnan area. By integrating advanced Hyper-Parameter Optimization (HPO) techniques, such as Optuna from machine learning, an inversion model was developed, incorporating the bands of Sentinel-2 and Sentinel-3 as model features. Among these features, bands with lower spatial resolution are employed to furnish surrounding information, thereby enhancing the inversion accuracy. The research outcomes demonstrate that: 1) The model constructed based on the HPO method, Optuna, attained favourable inversion results, with R2 values of 0.68, 0.77, 0.35, and 0.60 for Total Nitrogen (TN), Total Phosphorus (TP), Ammonia Nitrogen (NH3-N), and Chemical Oxygen Demand (COD), respectively. 2) The fusion of Sentinel-2 and Sentinel-3 data enhanced the inversion accuracy compared to using them separately, highlighting the considerable significance of multi-source data fusion methods in improving inversion accuracy. This research fills a void in the remote sensing inversion domain and lays the groundwork for future endeavours. [ABSTRACT FROM AUTHOR]

To enhance the stability of photovoltaic power grid integration and improve power prediction accuracy, a photovoltaic power prediction method based on an improved snow ablation optimization algorithm (Good Point and Vibration Snow Ablation Optimizer, GVSAO) and Bi-directional Long Short-Term Memory (Bi-LSTM) network is proposed. Weather data is divided into three typical categories using K-means clustering, and data normalization is performed using the minmax method. The key structural parameters of Bi-LSTM, such as the feature dimension at each time step and the number of hidden units in each LSTM layer, are optimized based on the Good Point and Vibration strategy. A prediction model is constructed based on GVSAO-Bi-LSTM, and typical test functions are selected to analyze and evaluate the improved model. The research results show that the average absolute percentage error of the GVSAO-Bi-LSTM prediction model under sunny, cloudy, and rainy weather conditions are 4.75%, 5.41%, and 14.37%, respectively. Compared with other methods, the prediction results of this model are more accurate, verifying its effectiveness. [ABSTRACT FROM AUTHOR]

The glass transition temperature of polymers is a key parameter in meeting the application requirements for energy absorption. Previous studies have provided some data from slow, expensive trial-and-error procedures. By recognizing these data, machine learning algorithms are able to extract valuable knowledge and disclose essential insights. In this study, a dataset of 7174 samples was utilized. The polymers were numerically represented using two methods: Morgan fingerprint and molecular descriptor. During preprocessing, the dataset was scaled using a standard scaler technique. We removed the features with small variance from the dataset and used the Pearson correlation technique to exclude the features that were highly connected. Then, the most significant features were selected using the recursive feature elimination method. Nine machine learning techniques were employed to predict the glass transition temperature and tune their hyperparameters. The models were compared using the performance metrics of mean absolute error (MAE), root mean square error (RMSE), and coefficient of determination (R2). We observed that the extra tree regressor provided the best results. Significant features were also identified using statistical machine learning methods. The SHAP method was also employed to demonstrate the influence of each feature on the model's output. This framework can be adaptable to other properties at a low computational expense. [ABSTRACT FROM AUTHOR]

Recently, the world has been dealing with a severe outbreak of COVID-19. The rapid transmission of the virus causes mild to severe cases of cough, fever, body aches, organ failures, and death. An increasing number of patients, fewer diagnostic options, and extended waiting periods for test results all put pressure on healthcare systems, increasing the virus’s spread. A concise and accurate automatic diagnosis is crucial to identify infected patients in the early stage. This paper proposes a machine learning-based predictive framework to identify COVID-19 cases from clinical data using an optimized union ensemble feature selection (OUEFS) approach. The OUEFS is based on the union ensemble of the feature subsets obtained through a rigorous feature selection (FS) process. It also involves a performance optimization of the ML classifiers. Initially the OUEFS identified key features from the publicly accessible COVID-19 dataset using FS methods such as Mutual Information Feature Selection (MIFS), Recursive Feature Elimination (RFE), and the RidgeCV. The most important features were selected using Top-k thresholding technique. Then selected subsets of features were integrated using a union ensemble approach where an optimal combination of features with enhanced predictive power is derived. This composite feature set was subsequently utilized for model training and evaluation. The classification was conducted using machine learning algorithms such as linear SVM, gradient boosting (GB), logistic regression (LR), and Adaboost to compare their effectiveness on individual and combined feature subsets. We also conducted a Genetic Algorithm (GA) based hyperparameter optimization (HPO) which further refined our training process and enhanced the accuracy of our proposed approach. Experimental results show that the union ensemble of MIFS and RidgeCV FS techniques and the Adaboost classifier and GA HPO achieved 96.30% accuracy. Our optimized union ensemble approach demonstrated superior performance over previous ensemble-based approaches to predict COVID-19 disease, thus offering a robust tool for early and efficient diagnosis without requiring hospital visits.

Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving distributed Machine Learning (ML), enabling model training across distributed devices without compromising data privacy. However, the impact of hyper-parameters on FL model performance remains understudied and most of the existing FL studies rely on default or out-of-the-box hyper-parameters, often leading to suboptimal convergence. This study specifically investigates the intricate relationship between key hyper-parameters—learning rate, epochs per round, batch size, and client participation ratio (CPR)—and the performance of FL models on two distinct datasets: CIFAR-10 using ResNet-18 and FashionMNIST using a simple CNN model. Through systematic exploration on these datasets, employing a centralized server and 200 clients, we elucidate the significant impact of varying hyper-parameters. Our findings underscore the importance of dataset-specific hyper-parameter optimization, revealing contrasting optimal configurations for the complex CIFAR-10 dataset and the simpler FashionMNIST dataset. Additionally, the correlation analysis offers a deep understanding of hyper-parameter inter-dependencies, essential for effective optimization. This study provides valuable insights for practitioners to customize hyper-parameter configurations, ensuring optimal performance for FL models trained on different types of datasets and provides a foundation for future exploration in hyper-parameter optimization within the FL domain.

With the rapid development of Internet of Vehicles (IoV) technology, Intelligent Connected Vehicles (ICVs) have richer vehicle information functions and applications. In recent years, as ICVs have become more complex and intelligent, vehicle information security is facing great threats and challenges. Therefore, it is of great significance to develop efficient intrusion detection methods to protect the information security of IoV. In this paper, after analyzing the vulnerability of intra-vehicle networks (IVNs) and external vehicle networks (EVNs), we propose a lightweight intrusion detection method, which uses MobileNetv2 as the backbone, combines transfer learning (TL) techniques and the hyper-parameter optimization (HPO) method. The proposed method can detect various types of attacks, and the Accuracy, Precision, and Recall on the Car-Hacking dataset representing IVNs data are all 100 % . The Accuracy, Precision, and Recall on the CICIDS2017 dataset representing EVNs data are all 99.93 % . The average processing time of each packet tested is about 0.75 ms, and the model space is 23 M. Experimental results demonstrate that the proposed intrusion detection method is effective and lightweight. [ABSTRACT FROM AUTHOR]

XGBoost is well-known as an open-source software library that provides a regularizing gradient boosting framework. Although it is widely used in the machine learning field, its performance depends on the determination of hyper-parameters. This study focuses on the optimization algorithm for hyper-parameters of XGBoost by using Stochastic Schemata Exploiter (SSE). SSE, which is one of Evolutionary Algorithms, is successfully applied to combinatorial optimization problems. SSE is applied for optimizing hyper-parameters of XGBoost in this study. The original SSE algorithm is modified for hyper-parameter optimization. When comparing SSE with a simple Genetic Algorithm, there are two interesting features: quick convergence and a small number of control parameters. The proposed algorithm is compared with other hyper-parameter optimization algorithms such as Gradient Boosted Regression Trees (GBRT), Tree-structured Parzen Estimator (TPE), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), and Random Search in order to confirm its validity. The numerical results show that SSE has a good convergence property, even with fewer control parameters than other methods. [ABSTRACT FROM AUTHOR]

To enhance the stability of photovoltaic power grid integration and improve power prediction accuracy, a photovoltaic power prediction method based on an improved snow ablation optimization algorithm (Good Point and Vibration Snow Ablation Optimizer, GVSAO) and Bi-directional Long Short-Term Memory (Bi-LSTM) network is proposed. Weather data is divided into three typical categories using K-means clustering, and data normalization is performed using the minmax method. The key structural parameters of Bi-LSTM, such as the feature dimension at each time step and the number of hidden units in each LSTM layer, are optimized based on the Good Point and Vibration strategy. A prediction model is constructed based on GVSAO-Bi-LSTM, and typical test functions are selected to analyze and evaluate the improved model. The research results show that the average absolute percentage error of the GVSAO-Bi-LSTM prediction model under sunny, cloudy, and rainy weather conditions are 4.75%, 5.41%, and 14.37%, respectively. Compared with other methods, the prediction results of this model are more accurate, verifying its effectiveness.

In this paper, the self-attention layer of a graph convolutional neural network is first constructed to output the important information in the network structure. The migration learning network model is established, and the sample data are preprocessed and trained sequentially. The final processing results are used as the initial data for abnormal power consumption detection. Introduce Bayes’ theorem to optimize the hyperparameters of the model. The optimized model is applied in the abnormal power consumption detection system to identify abnormal power consumption events and provide specific processing solutions. Through the detection of the system, it was found that the voltage of the test user dropped from a 100V cliff to about 20V in late November, which was determined by the system to be a power consumption abnormality, and, therefore, an operation and maintenance order was issued. The site survey revealed that the data was in line with the system detection. Calculating the power consumption information of another user, the phase voltage of this user stays around 85-100V, far below 150V, so the undercounting of power is verified for the user, and the amount of power that should be recovered is 201.22kW.

Accurate pre-harvest crop yield estimation is vital for agricultural sustainability and economic stability. The existing yield estimating models exhibit deficiencies in insufficient examination of hyperparameters, lack of robustness, restricted transferability of meta-models, and uncertain generalizability when applied to agricultural data. This study presents a novel meta-knowledge-guided framework that leverages three diverse agricultural datasets and explores meta-knowledge transfer in frequent hyperparameter optimization scenarios. The framework’s approach involves base tasks using LightGBM and Bayesian Optimization, which automates hyperparameter optimization by eliminating the need for manual adjustments. Conducted rigorous experiments to analyze the meta-knowledge transformation of RGPE, SGPR, and TransBO algorithms, achieving impressive R2 values (0.8415, 0.9865, 0.9708) using rgpe_prf meta-knowledge transfer on diverse datasets. Furthermore, the framework yielded excellent results for mean squared error (MSE), mean absolute error (MAE), scaled MSE, and scaled MAE. These results emphasize the method’s significance, offering valuable insights for crop yield estimation, benefiting farmers and the agricultural sector.

Quantitative precipitation nowcasting QPN is a powerful tool with many applications, such as rainfall prediction, urban sewage control, etc. Recently, researchers have focused on using machine learning and deep learning models for nowcasting raindrops. A few famous frameworks for nowcasting raindrops in a short leading time are Rainymotion, Segnet, Unet and RainNet. Rainymotion was developed using machine learning models and extracting areas of interest. Segnet is a deep, fully convolutional neural network architecture for semantic pixel-wise segmentation with a core trainable segmentation engine consisting of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer while the U-net is a radar data-based model for precipitation nowcasting. The U-Net architecture is based on convolutional neural networks CNNs. The three models, RainyMotion, U-Net, and Segnet, are used to compare the experiment results. In this manuscript, we combined the genetic algorithm GA with the RainNet model to improve the nowcasting performance of the model RainNet. The GA is used to tune hyperparameters such as filter size, activation function, and the number of epochs for the model. To enhance the performance of the baseline models for nowcasting, an objective function is defined to decrease the mean absolute error MAE below 0.3 for the next 5 minutes. The proposed algorithm was evaluated on radar images taken by the German Weather Service DWD, a German Meteorological Service based in Offenbach am Main, a city in Hesse, Germany, which monitors weather and meteorological conditions over Germany. The result from our experiments obtained indicated that the RainNet+GA model is more accurate in predicting short-term raindrops but requires an expensive implementation of a powerful computer to run on Graphical processor Unit GPU. Our hybrid model RainNet +GA did not overcome the computational power of using GPU. However, the experimental results with a loss of 0.021 showed a much smaller mean absolute error value for the next 5 minutes compared to other models and the original RainNet structure. With the use of GA, it has been proven that using novel developed activation functions such as Scaled Exponential Linear Units' SELU' and 'Swish' is more desirable than the rectified linear activation function ReLU. [ABSTRACT FROM AUTHOR]

Floods are both complex and destructive, and in most parts of the world cause injury, death, loss of agricultural land, and social disruption. Flood susceptibility (FS) maps are used by land‐use managers and land owners to identify areas that are at risk from flooding and to plan accordingly. This study uses machine learning ensembles to produce objective and reliable FS maps for the Haraz watershed in northern Iran. Specifically, we test the ability of the support vector regression (SVR), together with linear kernel (LK), base classifier (BC), and hyper‐parameter optimization (HPO), to identify flood‐prone areas in this watershed. We prepared a map of 201 past floods to predict future floods. Of the 201 flood events, 151 (75%) were used for modeling and 50 (25%) were used for validation. Based on the relevant literature and our field survey of the study area, 10 effective factors were selected and prepared for flood zoning. The results show that three of the 10 factors are most important for predicting flood‐sensitive areas, specifically and in order of importance, slope, distance to the river and river. Additionally, the SVR‐HPO model, with area under the curve values of 0.986 and 0.951 for the training and testing phases, outperformed the other two tested models. [ABSTRACT FROM AUTHOR]

Background: Pigmented skin lesions (PSLs) pose medical and esthetic challenges for those affected. PSLs can cause skin cancers, particularly melanoma, which can be life‐threatening. Detecting and treating melanoma early can reduce mortality rates. Dermoscopic imaging offers a noninvasive and cost‐effective technique for examining PSLs. However, the lack of standardized colors, image capture settings, and artifacts makes accurate analysis challenging. Computer‐aided diagnosis (CAD) using deep learning models, such as convolutional neural networks (CNNs), has shown promise by automatically extracting features from medical images. Nevertheless, enhancing the CNN models' performance remains challenging, notably concerning sensitivity. Materials and methods: In this study, we aim to enhance the classification performance of selected pretrained CNNs. We use the 2019 ISIC dataset, which presents eight disease classes. To achieve this goal, two methods are applied: resolution of the dataset imbalance challenge through augmentation and optimization of the training hyperparameters via Bayesian tuning. Results: The performance improvement was observed for all tested pretrained CNNs. The Inception‐V3 model achieved the best performance compared to similar results, with an accuracy of 96.40% and an AUC of 0.98. Conclusion: According to the study, classification performance was significantly enhanced by augmentation and Bayesian hyperparameter tuning. [ABSTRACT FROM AUTHOR]

Severe space weather produced by disturbed conditions on the Sun results in harmful effects both for humans in space and in high‐latitude flights, and for technological systems such as spacecraft or communications. Also, geomagnetically induced currents (GICs) flowing on long ground‐based conductors, such as power networks, potentially threaten critical infrastructures on Earth. The first step in developing an alarm system against GICs is to forecast them. This is a challenging task given the highly non‐linear dependencies of the response of the magnetosphere to these perturbations. In the last few years, modern machine‐learning models have shown to be very good at predicting magnetic activity indices. However, such complex models are on the one hand difficult to tune, and on the other hand they are known to bring along potentially large prediction uncertainties which are generally difficult to estimate. In this work we aim at predicting the SYM‐H index characterizing geomagnetic storms multiple‐hour ahead, using public interplanetary magnetic field (IMF) data from the Sun‐Earth L1 Lagrange point and SYM‐H data. We implement a type of machine‐learning model called long short‐term memory (LSTM) network. Our scope is to estimate the prediction uncertainties coming from a deep‐learning model in the context of forecasting the SYM‐H index. These uncertainties will be essential to set reliable alarm thresholds. The resulting uncertainties turn out to be sizable at the critical stages of the geomagnetic storms. Our methodology includes as well an efficient optimization of important hyper‐parameters of the LSTM network and robustness tests. Plain Language Summary: Geomagnetic storms are disturbances of the geomagnetic field caused by interactions between the solar wind and particle populations mainly in the Earth's magnetosphere. These time‐varying magnetic fields induce electrical currents on long ground‐based conductors that can damage power transmission grids and other critical infrastructures on Earth. As a first step to forecast the ground magnetic perturbations caused by geomagnetic storms at specific mid‐latitude locations, the objective of this work is to predict the SYM‐H activity index, which is generated from ground observations of the geomagnetic field at low and mid‐latitudes, and which provides a measure of the strength and duration of geomagnetic storms. We use the IMF data measured by the Advanced Composition Explorer spacecraft at the L1 Lagrangian point and SYM‐H values to forecast the behavior and severity of geomagnetic storms multiple‐hour ahead. This forecasting is done using a type of artificial neural network model called long short‐term memory. We also propose robust ways to estimate the uncertainties of these predictions, which help us to better understand machine‐learning models in activity indices prediction and lead to more accurate and reliable forecasting of geomagnetic storms and their ground effects in the near future. Key Points: A long short‐term memory (LSTM) model is built to forecast SYM‐H index multiple‐hour ahead using interplanetary magnetic field (IMF) measurements and SYM‐H valuesPrediction uncertainties from the LSTM model are estimated and turn out to be considerable in the critical phases of geomagnetic stormsThe uncertainty quantification is found to be crucial to achieve a reliable forecasting model and determine the optimal look‐forward [ABSTRACT FROM AUTHOR]

The Stochastic Schemata Exploiter (SSE), one of the Evolutionary Algorithms, is designed to find the optimal solution of a function. SSE extracts common schemata from individual sets with high fitness and generates individuals from the common schemata. For hyper-parameter optimization, the initialization method, the schema extraction method, and the new individual generation method, which are characteristic processes in SSE, are extended. In this paper, an SSE-based multi-objective optimization for AutoML is proposed. AutoML gives good results in terms of model accuracy. However, if only model accuracy is considered, the model may be too complex. Such complex models cannot always be allowed because of the long computation time. The proposed method maximizes the stacking model accuracy and minimizes the model complexity simultaneously. When compared with existing methods, SSE has interesting features such as fewer control parameters and faster convergence properties. The visualization method makes the optimization process transparent and helps users understand the process. [ABSTRACT FROM AUTHOR]

Abstract Floods are both complex and destructive, and in most parts of the world cause injury, death, loss of agricultural land, and social disruption. Flood susceptibility (FS) maps are used by land‐use managers and land owners to identify areas that are at risk from flooding and to plan accordingly. This study uses machine learning ensembles to produce objective and reliable FS maps for the Haraz watershed in northern Iran. Specifically, we test the ability of the support vector regression (SVR), together with linear kernel (LK), base classifier (BC), and hyper‐parameter optimization (HPO), to identify flood‐prone areas in this watershed. We prepared a map of 201 past floods to predict future floods. Of the 201 flood events, 151 (75%) were used for modeling and 50 (25%) were used for validation. Based on the relevant literature and our field survey of the study area, 10 effective factors were selected and prepared for flood zoning. The results show that three of the 10 factors are most important for predicting flood‐sensitive areas, specifically and in order of importance, slope, distance to the river and river. Additionally, the SVR‐HPO model, with area under the curve values of 0.986 and 0.951 for the training and testing phases, outperformed the other two tested models.

Federated Learning is a new approach for distributed training of a deep learning model on data scattered across a large number of clients while ensuring data privacy. However, this approach faces certain limitations, including a longer convergence time compared to typical deep learning models. Existing federated optimization algorithms often employ the same hyper-parameters for all clients, disregarding potential system heterogeneity and varying local data availability, which contributes to an even longer convergence time and more communication rounds. To address this challenge, we propose FedHPO, a new federated optimization algorithm that adaptively modifies hyper-parameters of each client’s local model during training, such as learning rate and epochs. This adaptability facilitates quicker convergence of the client’s local model, which in turn helps the global model converge faster, consequently reducing overall convergence time and the required communication rounds. In addition, FedHPO does not require any additional complexity since each client adjusts hyper-parameters independently based on the training results obtained in each epoch. In our evaluation, we compare FedHPO with other algorithms, such as FedAVG, FedAVGM, FedProx, and FedYogi, using both IID and non-IID distributed datasets. The results demonstrate the promising outcomes of FedHPO, showcasing reduced convergence time and fewer required communication rounds in comparison to alternative algorithms.

In machine learning, hyperparameter tuning is strongly useful to improve model performance. In our research, we concentrate our attention on classifying imbalanced data by cost-sensitive support vector machines. We propose a multi-objective approach that optimizes model's hyper-parameters. The approach is devised for imbalanced data. Three SVM model's performance measures are optimized. We present the algorithm in a basic version based on genetic algorithms, and as an improved version based on genetic algorithms combined with decision trees. We tested the basic and the improved approach on benchmark datasets either as serial and parallel version. The improved version strongly reduces the computational time needed for finding optimized hyper-parameters. The results empirically show that suitable evaluation measures should be used in assessing the classification performance of classification models with imbalanced data. [ABSTRACT FROM AUTHOR]

We present an adaptive deep collocation method (DCM) based on physics-informed deep learning for the melting heat transfer analysis of a non-Newtonian (Sisko) fluid over a moving surface with nonlinear thermal radiation. Fitted neural network search (NAS) and model based transfer learning (TL) are developed to improve model computational efficiency and accuracy. The governing equations for this boundary-layer flow problem are derived using Buongiorno's and a nonlinear thermal radiation model. Next, similarity transformations are introduced to reduce the governing equations into coupled nonlinear ordinary differential equations (ODEs) subjected to asymptotic infinity boundary conditions. By incorporating physics constraints into the neural networks, we employ the proposed deep learning model to solve the coupled ODEs. The imposition of infinity boundary conditions is carried out by adding an inequality constraint to the loss function, with infinity added to the hyper-parameters of the neural network, which is updated dynamically in the optimization process. The effects of various dimensionless parameters on three profiles (velocity, temperature, concentration) are investigated. Finally, we demonstrate the performance and accuracy of the adaptive DCM with transfer learning through several numerical examples, which can be the promising surrogate model to solve boundary layer problems. [ABSTRACT FROM AUTHOR]

Cardiovascular diseases, globally recognized as prominent contributors to morbidity and mortality, have led to an imperative demand for precise, accessible, and efficient diagnostic methodologies. This study introduces a hybrid classification system integrating an ensemble model and a Fuzzy C Means-based neural network with the objective of augmenting predictive accuracy. A comparative analysis on scalar standards was undertaken to determine the optimal feature scaling technique, thereby enhancing predictive proficiency while optimizing time efficiency. The study further incorporates Random Forest, Support Vector Machines, k-Nearest Neighbor, and deep learning models into the diagnostic framework, while employing a confusion matrix as a performance evaluation tool. The GridsearchCV technique is utilized for hyperparameter optimization, its influence on the accuracy of machine learning (ML) models is critically examined. Special attention is given to the role of outliers and their manipulation using supervised ML algorithms, investigating the impact of outlier exclusion on model accuracy. The experimental data was sourced from a cardiovascular patients dataset in the UCI Machine Learning Repository. The findings of the study suggest that the proposed classifier ensemble model surpasses comparable advancements, achieving an exemplary classification accuracy of 98.78%. This paper thus contributes to the evolving landscape of ML application in cardiovascular disease prediction, emphasizing the significance of outlier detection and hyperparameter optimization. [ABSTRACT FROM AUTHOR]

Training and validation of Neural Networks (NN) are very computationally intensive. In this paper, we propose a distributed system based NN infrastructure that achieves two goals: to accelerate model training, specifically for hyperparameter optimization, and to re-use some of these intermediate models to evaluate the uncertainty of the model. By accelerating model training, we can obtain a large set of potential models and compare them in a shorter amount of time. Automating this process reduces development time and provides an easy way to compare different models. Our application runs different models on distinct servers with a single training data set, each with tweaked hyperparameters. By adding uncertainty to our results, our framework provides not just a single prediction but a distribution over predictions. Adding uncertainty is essential to some NN applications since most models assume that the input data distributions are identical between test and validation. However, in reality, they are producing some catastrophic mistakes. Since our solution is a distributed system, we make our implementation robust to common distributed system failures (servers going down, loss of communication among some nodes, and others). Furthermore, we use a gossip-style heartbeat protocol for failure detection and recovery. Finally, some preliminary results using a black-box approach to generate the training models show that our infrastructure scales well in different hardware platforms. [ABSTRACT FROM AUTHOR]

The number of studies in the literature that diagnose cancer with machine learning using genome data is quite limited. These studies focus on the prediction performance, and the extraction of genomic factors that cause disease is often overlooked. However, finding underlying genetic causes is very important in terms of early diagnosis, development of diagnostic kits, preventive medicine, etc. The motivation of our study was to diagnose bladder cancer (BCa) based on genetic data and to reveal underlying genetic factors by using machine-learning models. In addition, conducting hyper-parameter optimization to get the best performance from different models, which is overlooked in most studies, was another objective of the study. Within the framework of these motivations, C4.5, random forest (RF), artificial neural networks (ANN), and deep learning (DL) were used. In this way, the diagnostic performance of decision tree (DT)-based models and black box models on BCa was also compared. The most successful model, DL, yielded an area under the curve (AUC) of 0.985 and a mean square error (MSE) of 0.069. For each model, hyper-parameters were optimized by an evolutionary algorithm. On average, hyper-parameter optimization increased MSE, root mean square error (RMSE), LogLoss, and AUC by 30%, 17.5%, 13%, and 6.75%, respectively. The features causing BCa were extracted. For this purpose, entropy and Gini coefficients were used for DT-based methods, and the Gedeon variable importance was used for black box methods. The single nucleotide polymorphisms (SNPs) rs197412, rs2275928, rs12479919, rs798766 and rs2275928, whose BCa relations were proven in the literature, were found to be closely related to BCa. In addition, rs1994624 and rs2241766 susceptibility loci were proposed to be examined in future studies. [ABSTRACT FROM AUTHOR]

Cyberbullying has emerged as a pervasive issue in the digital age, necessitating advanced techniques for effective detection and mitigation. This research explores the integration of word embeddings, emotional features, and federated learning to address the challenges of centralized data processing and user privacy concerns prevalent in previous methods. Word embeddings capture semantic relationships and contextual information, enabling a more nuanced understanding of text data, while emotional features derived from text extend the analysis to encompass the affective dimension, enhancing cyberbullying identification. Federated learning, a decentralized learning paradigm, offers a compelling solution to centralizing sensitive user data by enabling collaborative model training across distributed devices, preserving privacy while harnessing collective intelligence. In this study, we conduct an in-depth investigation into the fusion of word embeddings, emotional features, and federated learning, complemented by the utilization of BERT, Convolutional Neural Networks (CNN), Deep Neural Networks (DNN), and Long Short-Term Memory (LSTM) models. Hyperparameters and neural architecture are explored to find optimal configurations, leading to the generation of superior results. These techniques are applied in the context of cyberbullying detection, using publicly available multi-platform (social media) cyberbullying datasets. Through extensive experiments and evaluations, our proposed framework demonstrates superior performance and robustness compared to traditional methods. The results illustrate the enhanced ability to identify and combat cyberbullying incidents effectively, contributing to the creation of safer online environments. Particularly, the BERT model consistently outperforms other deep learning models (CNN, DNN, LSTM) in cyberbullying detection while preserving the privacy of local datasets for each social platform through our improved federated learning setup. We have provided Differential Privacy based security analysis for the proposed method to further strengthen the privacy and robustness of the system. By leveraging word embeddings, emotional features, and federated learning, this research opens new avenues in cyberbullying research, paving the way for proactive intervention and support mechanisms. The comprehensive approach presented herein highlights the substantial strengths and advantages of this integrated methodology, setting a foundation for future advancements in cyberbullying detection and mitigation.

In this paper, we address the challenge of limited resources in Internet of Things (IoT) devices by proposing a solution based on digital twin in distributed edge computing networks. Edge computing is a promising approach that moves computing resources closer to the network’s edge to reduce response times in IoT applications. However, simply offloading tasks from IoT devices to edge computing does not accelerate user control. To enhance task performance and improve user management experience, we introduce optimized task scheduling and virtual object management based on a digital twin concept. Our system incorporates virtualization, synchronization, visualization, and simulation functionalities to provide digital twin capabilities. Additionally, we develop a user-friendly web application with a graphical user interface (GUI) for intuitive management of edge computing services. To support our approach, we implement an edge computing supervisor that generates virtualized objects such as edge gateways, IoT devices, and services. These virtual objects serve as resources for creating tasks. Using our proposed digital twin platform, users can dynamically create new tasks based on demand, easily deploy and execute tasks in specific locations, and dynamically allocate edge network resources according to task requirements. An optimized task scheduling mathematical model is presented to compare task scheduling done with and without optimization. Further, the edge computing and digital twin based optimized task scheduling method is integrated with Federated Learning for collaborative learning and privacy preserved computation of sensors sensitive data. We demonstrate the effectiveness of our system by generating tasks for data collection related to indoor environment for prediction of Predicted Mean Vote (PMV) for thermal comfort index of smart homes occupants using HTTP and IoTivity-based devices in distributed edge computing networks. These tasks are properly delivered and executed on the expected edge gateways, showcasing the successful integration of our digital twin platform with edge computing networks. Further, the optimized task scheduling has improved the overall performance of the proposed system, keeping in view latency and processing time.

Simulation and optimization methods have been widely used in forklift design due to their cost-effectiveness. However, this type of method involves challenges such as the accuracy of the simulation model and the simulation solution time. These challenges reduce the stability and precision of the surrogate model and hence generate further optimization errors. In this paper, a multi-objective surrogate modeling (MSM) method for telescopic boom forklifts based on closed-loop transfer learning is proposed in order to solve these challenges. The MSM consists of the following two steps: to pre-train an initial deep neural network model (deep model) with a large amount of existing simulation data from the same type of forklift and to transfer the model with a small amount of measurement data collected on the current forklift. A general framework for deep neural network (DNN) training is introduced to improve the approximation ability of the initial model. Moreover, a novel uncertainty-analysis-based sampling method is suggested for measurement data development, and combined with transfer learning to form a closed-loop mode to improve the stability of the final model. The superiority of MSM is demonstrated through comparative studies with the fine-tuning method on a telescopic boom forklift with two objectives. The experimental results show that the Correlation coefficient (R) of the deep model can reach 0.9971 by using only 80 sets of training data. In addition, it can also achieve an improvement of at least a 13.25% reduction in Root Mean Squared Error (RMSE) and a 9.19% reduction on average in Maximum Absolute Error (MAE), as well as stronger robustness compared to the benchmarks. Furthermore, it will provide a valuable reference for the simulation optimization of complex electromechanical products.

Colorectal cancer (CRC) is one of the most common and malignant types of cancer worldwide. Colonoscopy, considered the gold standard for CRC screening, allows immediate removal of polyps, which are precursors to CRC. Many computer-aided diagnosis systems (CADs) have been proposed for automatic polyp detection. Most of these systems are based on traditional machine learning algorithms and their generalization ability, sensitivity and specificity are limited. On the other hand, with the widespread use of deep learning algorithms in medical image analysis and the successful results in the analysis of colonoscopy images, especially in the early and accurate detection of polyps, these problems are eliminated in recent years. In short, deep learning algorithms and applications have gained a critical role in CAD systems for real-time autonomous polyp detection. Here, we make significant improvements to object detection algorithms to improve the performance of CAD-based real-time polyp detection systems. We integrate the artificial bee colony algorithm (ABC) into the YOLO algorithm to optimize the hyper-parameters of YOLO-based algorithms. The proposed method can be easily integrated into all YOLO algorithms such as YOLOv3, YOLOv4, Scaled-YOLOv4, YOLOv5, YOLOR and YOLOv7. The proposed method improves the performance of the Scaled-YOLOv4 algorithm with an average of more than 3% increase in mAP and a more than 2% improvement in F1 value. In addition, the most comprehensive study is conducted by evaluating the performance of all existing models in the Scaled-YOLOv4 algorithm (YOLOv4s, YOLOv4m, YOLOV4-CSP, YOLOv4-P5, YOLOV4-P6 and YOLOv4-P7) on the novel SUN and PICCOLO polyp datasets. The proposed method is the first study for the optimization of YOLO-based algorithms in the literature and makes a significant contribution to the detection accuracy. [ABSTRACT FROM AUTHOR]

Accurate prediction of soil moisture content in tea plantations plays a crucial role in optimizing irrigation practices and improving crop productivity. Traditional methods for SMC prediction are difficult to implement due to high costs and labor requirements. While machine learning models have been applied, their performance is often limited by the lack of sufficient data. To address the challenges of inaccurate and inefficient soil moisture prediction in tea plantations and enhance predictive performance, an improved support-vector-machine- (SVM) based model was developed to predict the SMC in a tea plantation. The proposed model addresses several limitations of existing approaches by incorporating novel features and enhancing the SVM algorithm's performance, which was improved with the Bald Eagle Search algorithm (BES) method for hyper-parameter optimization. The study utilized a comprehensive dataset comprising soil moisture measurements and relevant environmental variables collected from a tea plantation. Feature selection techniques were applied to identify the most informative variables, including rainfall, temperature, humidity, and soil type. The selected features were then used to train and optimize the SVM model. The proposed model was applied to prediction of soil water moisture in a tea plantation in Guangxi State-owned Fuhu Overseas Chinese Farm. Experimental results demonstrated the superior performance of the improved SVM model in predicting soil moisture content compared to traditional SVM approaches and other machine-learning algorithms. The model exhibited high accuracy, robustness, and generalization capabilities across different time periods and geographical locations with R2, MSE, and RMSE of 0.9435, 0.0194 and 0.1392, respectively, which helps to enhance the prediction performance, especially when limited real data are available. The proposed SVM-based model offers several advantages for tea plantation management. It provides timely and accurate soil moisture predictions, enabling farmers to make informed decisions regarding irrigation scheduling and water resource management. By optimizing irrigation practices, the model helps enhance tea crop yield, minimize water usage, and reduce environmental impact. [ABSTRACT FROM AUTHOR]

Accurate bearing fault classification is essential for the safe and stable operation of rotating machinery. The success of Machine Learning (ML) in fault classification is mainly dependent on efficient features and the optimal pre-defined hyper-parameters. Various hyper-parameter optimization (HPO) methods have been proposed to tune the ML algorithms' hyper-parameters in low dimensions but ignore the hyper-parameters of Feature Engineering (FE). The hyper-parameter dimension is high because both FE and the ML algorithm contain many hyper-parameters. This paper proposed a new HPO method for high dimensions based on dimension reduction and partial dependencies. Firstly, the whole hyper-parameter space is separated into two subspaces of FE and the ML algorithm to reduce time consumption. Secondly, the sensitive intervals of hyperparameters can be recognized by partial dependencies due to the nonlinearity of the relationship between the hyperparameters. Then HPO is conducted in intervals to acquire more satisfactory accuracy. The proposed method is verified on three OpenML datasets and the CWRU bearing dataset. The results show that it can automatically construct efficient domain features and outperforms traditional HPO methods and famous ML algorithms. The proposed method is also very time efficient. [ABSTRACT FROM AUTHOR]

Hyper-parameter optimization (HO), regardless of the type of optimization, inherently not only increases the completion time of the algorithm to be optimized but also creates a remarkable computational burden. However, employing the most suitable HO technique for a specific problem may not be sufficient to improve the performance of the selected machine learning algorithm. In such cases, it is common to deploy default values of the initialization hyper-parameters of HO. Instead, a configured set of initialization hyper-parameters of HO is significantly more impactful than a default mode of HO. In this study, a metaheuristic ensemble technique is proposed to configure the initialization hyper-parameters of HO. The proposed method is devised after an extensive time analysis of metaheuristics and applied to Echo State Network (ESN). The experiment performed with weather forecast data shows that metaheuristic initialization methods are quite compatible with evolutionary algorithms. In the benchmark, the proposed method outperformed two alternatives. Probabilistic methods such as Bayesian optimization are not preferable for metaheuristic initialization methods, according to the results of the experiment. Metaheuristic hyper-parameter initialization methods can be performed by utilizing Random search that provides a moderate performance in which there are hardware-restricted sources. Last, the hyper-parameter called leakingrate of ESN is the most sensitive one and creates the largest churns in the prediction performance. [ABSTRACT FROM AUTHOR]

Soil moisture is a crucial factor in the field of meteorology, hydrology, and agricultural sciences. In agricultural production, surface soil moisture (SSM) is crucial for crop yield estimation and drought monitoring. For SSM inversion, a synthetic aperture radar (SAR) offers a trustworthy data source. However, for agricultural fields, the use of SAR data alone to invert SSM is susceptible to the influence of vegetation cover. In this paper, based on Sentinel-1 microwave remote sensing data and Sentinel-2 optical remote sensing data, a convolution neural network optimized by sparrow search algorithm (SSA-CNN) was suggested to invert farmland SSM. The feature parameters were first extracted from pre-processed remote sensing data. Then, the correlation analysis between the extracted feature parameters and field measured SSM data was carried out, and the optimal combination of feature parameters for SSM inversion was selected as the input data of the subsequent models. To enhance the performance of the CNN, the hyper-parameters of CNN were optimized using SSA, and the SSA-CNN model was built for SSM inversion based on the obtained optimal hyper-parameter combination. Three typical machine learning approaches, including generalized regression neural network, random forest, and CNN, were used for comparison to show the efficacy of the suggested method. With an average coefficient of determination of 0.80, an average root mean square error of 2.17 vol.%, and an average mean absolute error of 1.68 vol.%, the findings demonstrated that the SSA-CNN model with the optimal feature combination had a better accuracy among the 4 models. In the end, the SSM of the study region was inverted throughout four phenological periods using the SSA-CNN model. The inversion results indicated that the suggested method performed well in local situations. [ABSTRACT FROM AUTHOR]

Autism Spectrum Disorder (ASD) is a neurological impairment condition that severely impairs cognitive, linguistic, object recognition, interpersonal, and communication skills. Its main cause is genetic, and early treatment and identification can reduce the patient's expensive medical costs and lengthy examinations. We developed a machine learning (ML) architecture that is capable of effectively analysing autistic children's datasets and accurately classifying and identifying ASD traits. We considered the ASD screening dataset of toddlers in this study. We utilised the SMOTE method to balance the dataset, followed by feature transformation and selection methods. Then, we utilised several classification techniques in conjunction with a hyperparameter optimisation approach. The AdaBoost method yielded the best results among the classifiers. We employed ML and statistical approaches to identify the most crucial characteristics for the rapid recognition of ASD patients. We believe our proposed framework could be useful for early diagnosis and helpful for clinicians. [ABSTRACT FROM AUTHOR]

The choice of learning rate (LR) functions and policies has evolved from a simple fixed LR to the decaying LR and the cyclic LR, aiming to improve the accuracy and reduce the training time of Deep Neural Networks (DNNs). This article presents a systematic approach to selecting and composing an LR policy for effective DNN training to meet desired target accuracy and reduce training time within the pre-defined training iterations. It makes three original contributions. First, we develop an LR tuning mechanism for auto-verification of a given LR policy with respect to the desired accuracy goal under the pre-defined training time constraint. Second, we develop an LR policy recommendation system (LRBench) to select and compose good LR policies from the same and/or different LR functions through dynamic tuning, and avoid bad choices, for a given learning task, DNN model, and dataset. Third, we extend LRBench by supporting different DNN optimizers and show the significant mutual impact of different LR policies and different optimizers. Evaluated using popular benchmark datasets and different DNN models (LeNet, CNN3, ResNet), we show that our approach can effectively deliver high DNN test accuracy, outperform the existing recommended default LR policies, and reduce the DNN training time by 1.6-6.7× to meet a targeted model accuracy. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Ordnance Equipment Engineering is the property of Chongqing University of Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Recent advances in machine learning, specifically transformer architecture, have led to significant advancements in commercial domains. These powerful models have demonstrated superior capability to learn complex relationships and often generalize better to new data and problems. This paper presents a novel transformer-powered approach for enhancing prediction accuracy in multi-modal output scenarios, where sparse experimental data is supplemented with simulation data. The proposed approach integrates transformer-based architecture with a novel graph-based hyper-parameter optimization technique. The resulting system not only effectively reduces simulation bias, but also achieves superior prediction accuracy compared to the prior method. We demonstrate the efficacy of our approach on inertial confinement fusion experiments, where only 10 shots of real-world data are available, as well as synthetic versions of these experiments.

The increased usage of cyber-physical systems (CPS) has gained the focus of cybercriminals, particularly with the involvement of the internet, provoking an increased attack surface. The increased usage of these systems generates heavy data flows, which must be analyzed to ensure security. In particular, machine learning (ML) and deep learning (DL) algorithms have shown feasibility and promising results to fulfill the security requirement through the adoption of intelligence. However, the performance of these models strongly depends on the model structure, hyper-parameters, dataset, and application. So, the developers only possess control over defining the model structure and its hyper-parameters for diversified applications. Generally, not all models perform well in default hyper-parameter settings. Their specification is a challenging and complex task and requires significant expertise. This problem can be mitigated by utilizing hyper-parameter optimization (HPO) techniques, which intend to automatically find efficient learning model hyper-parameters in specific applications or datasets. This paper proposes an enhanced intelligent security mechanism for CPS by utilizing HPO. Specifically, exhaustive HPO techniques have been considered for performance evaluation and evaluation of computational requirements to analyze their capabilities to build an effective intelligent security model to cope with security infringements in CPS. Moreover, we analyze the capabilities of various HPO techniques, normalization, and feature selection. To ensure the HPO, we evaluated the effectiveness of a DL-based artificial neural network (ANN) on a standard CPS dataset under manual hyper-parameter settings and exhaustive HPO techniques, such as random search, directed grid search, and Bayesian optimization. We utilized the min-max algorithm for normalization and SelectKBest for feature selection. The HPO techniques performed better than the manual hyper-parameter settings. They achieved an accuracy, precision, recall, and F1 score of more than 98%. The results highlight the importance of HPO for performance enhancement and reduction of computational requirements, human efforts, and expertise. [ABSTRACT FROM AUTHOR]