Search

This article introduces a new iterative approach to explainable feature learning. During each iteration, new features are generated, first by applying arithmetic operations on the input set of features. These are then evaluated in terms of probability distribution agreements between values of samples belonging to different classes. Finally, a graph-based approach for feature selection is proposed, which allows for selecting high-quality and uncorrelated features to be used in feature generation during the next iteration. As shown by the results, the proposed method improved the accuracy of all tested classifiers, where the best accuracies were achieved using random forest. In addition, the method turned out to be insensitive to both of the input parameters, while superior performances in comparison to the state of the art were demonstrated on nine out of 15 test sets and achieving comparable results in the others. Finally, we demonstrate the explainability of the learned feature representation for knowledge discovery.

This article conducts a comparative analysis to investigate the effects of different classification algorithms and structural proximity indexes on the performance of the supervised link prediction approach to anticipating technology convergence at different forecast horizons. For this, we identify relationships between technologies of interest for different time periods and compute 10 structural proximity indexes among unconnected technologies at each period. We develop a set of classification models that identify potential convergence among unconnected technologies where each model is configured differently by a classification algorithm and a combination of the proximity indexes. We compare the performance of the classification models to investigate effective combinations of classification algorithms and proximity indexes at different forecast horizons. The empirical analysis on Wikipedia articles about artificial intelligence technology indicates that random forest outperforms others in short-term forecasting while support vector machine outperforms others in mid-term forecasting. We also identify structural proximity indexes that produce higher performance when combined with the most effective algorithm at each forecast horizon. The results of this article are expected to offer guidelines for choosing classification algorithms and indexes when applying the supervised link prediction approach in anticipating technology convergence.

Communication is the basic need of humans to interact with their surroundings. BCI acts as a great tool for providing the means of basic communication to people with locked-in syndrome. For such patients, these basic things matter most and give them a sense of independence. Communication through BCI can be done using various paradigms, P300 speller is one of these paradigms in which the user is instructed to focus their attention on the desired character and able to produce negative and positive slow cortical potential changes which is then interpreted by the BCI system. Although P300 based BCI was introduced over twenty years ago, the past few years have seen a rapid increase in P300 based BCI research. In this paper, we overview the current status of P300 BCI technology and also discussed and compare different approaches using Random Forest, Support Vector Machine (SVM), and Extreme Gradient Boosting (XgBoost) classification algorithm.

The focal role of flow in promoting desirable outcomes in companies, such as increased employees' well-being and performance, led scholars to study flow in the context of work. However, current measurement approaches which assess flow via self-report scales after task execution are limited due to obtrusiveness and a lack of real-time support. Hence, new measurement approaches must be created to overcome these limitations. In this paper, we use cardiac features (heart rate variability; HRV) and a Random Forest classifier to distinguish high and low flow. Our results from a large-scale lab experiment with 158 participants and a field study with nine participants reveal, that with HRV features alone, flow-classifiers can be built with an accuracy of 68.5% (lab) and 70.6% (field). Our research contributes to the challenge of developing a less obtrusive, real-time measurement method of flow based on physiological features and to investigate flow from a physiological perspective. Our findings may serve as foundation for future work aiming to build physio-adaptive systems which can improve employee's performance. For instance, these systems could ensure that no notifications are forwarded to employees while the system is 'sensing' flow.

With the tremendous advancement in machine learning and deep learning, organizations are using numerous algorithms for analyzing the huge amount of data to come up with insights which contains meaningful out comes. Especially in medical health care systems, machine learning is being used widely to forecast and treat the illness at early in stage. Cervical cancer is one of such diseases which can be diagnosed if identified in early stage, but the symptom-less nature of the disease has become challenging for the researchers and practitioners to predict the disease at early stage. There are numerous factors which will prone to cervical cancer risk in women. In this study we have proposed a stacked ensemble technique which takes heterogenous base learners and a meta learner for predicting the cervical cancer from various risk factors, SMOTE has been used for data Balancing and RFE with Random Forest used for feature extraction through which we have got better accuracy than the existing methods. We have also identified top 8 features which impact the advancement of the classification model.

Text Neck Syndrome is repeated stress injury to the neck, or it is a pain encountered in the neck due to excessive usage of handheld devices for a prolonged period. In this paper, we have used the Microsoft Kinect sensor to create the dataset for normal and abnormal sitting postures by obtaining 3D skeletal coordinates of people. Features are extracted from the data collected from the Kinect sensor data based on the various parameters like Minkowski distance, Euclidean distance, Manhattan distance, etc. Then we have applied different machine learning algorithms, namely, K-nearest neighbours, Random forest, Support vector machine, Logistic and Linear Regression, to classify them. The performance of various algorithms is analysed. The best testing accuracy achieved is 92.85% by using Random Forest and Support Vector Machine algorithms.

Distributed Denial of service (DDoS) attacks is an enormous threat to today’s cyber world, cyber networks are compromised by the attackers to distribute attacks in a large volume by denying the service to legitimate users. The toughest and challenging task in today’s network and network security engineers is to identify compromised traffic (attacked) and legitimate (normal) traffic. The main goal of the paper is to detect DDos attacks using classification algorithms. To achieve the goal the proposed system uses attacking tool to initiate attacks using Loic attacking tool with the data set extracted from open source tool Wireshark and transferring the dataset to apache Spark for detection analysis. The system also uses Apache spark machine learning algorithms (MLib), classification algorithms to classify the dataset. We use Naive Bayes, KNN and Random forest classification algorithms to classify normal traffic and attacked traffic. Our system is capable of detecting attacks with respect to any traffic protocols ICMP, TCP, or UDP. The accuracy of detection is compared on three classification algorithms and noted that random forest gives the accuracy of 96.75%.

This paper uses data sampled at hourly and daily frequencies to predict Bitcoin returns. We consider various advanced non-linear models based on a multitude of popular technical indicators that represent market trend, momentum, volume, and sentiment. We run a robust empirical exercise to observe the impact of forecast horizon, model type, time period, and the choice of inputs (predictors) on the forecast performance of the competing models. We find that Bitcoin prices are weakly efficient at the hourly frequency. In contrast, technical analysis combined with non-linear forecasting models becomes statistically significantly dominant relative to the random walk model on a daily horizon. Our comparative analysis identifies the random forest model as the most accurate at predicting Bitcoin. The estimated measures of the relative importance of predictors reveal that the nature of investing in the Bitcoin market evolved from trend-following to excessive momentum and sentiment in the most recent time period.

At present, under the condition of privacy-preserving, vertical federated learning (VFL) has played an important role in training the machine learning models in the application scenarios, such as medical prediction, fraud detection, in which the data is distributed vertically. Random forest is one of the most widely used machine learning methods in VFL, which has the advantages of strong predictive performance, availability, and the ability to parallel tasks. However, current research on privacy-preserving of vertical federated random forest is limited, and none of them can achieve the application level of security, that is, in a system where users are dynamically changing, not only privacy can be preserved, but also data integrity can be verified. Therefore, we propose a verifiable privacy-preserving scheme (VPRF) based on vertical federated random forest, in which the users are dynamic change. Firstly, we design homomorphic comparison and voting statistics algorithms based on multi-key homomorphic encryption for privacy preservation. Then, we propose a multi-client delegated computing verification algorithm to make up for the disadvantage that the above algorithms cannot verify data integrity. Finally, we used the datasets in UCI machine learning warehouse to evaluate the propose scheme. The experiment results indicate that our scheme is more efficient than the existing ones that can achieve the same security level.

Presurgical discrimination of 1p/19q codeletion status may have prognostic and diagnostic value for glioma patients for immediate personalized treatment. Artificial intelligence-based models have been proved as effective method to demonstrate computer aided diagnostic system for glioma cancer. An objective of study is to present an advanced biomedical texture descriptor to perform machine learning-assisted identification of 1p/19q codeletion status of low-grade glioma (LGG) cancer. An aim is to verify efficacy of textures, extracted using local binary pattern and derived from gray level co-occurrence matrix (GLCM). Proposed study used random forest-assisted radiomics model to analyse MRI images of 159 subjects. Four different advanced biomedical texture descriptors are proposed by experimenting different extensions of LBP method. These variants-(as variant I to IV) with 8-bit or 16-bit or 24-bit LBP codes are applied with different orientations in 5 × 5, 7 × 7 square-sized neighbourhood, which are recorded in LBP histograms. These histogram features are concatenated by GLCM-based textures including energy, correlation, contrast and homogeneity. Texture descriptors performed best with classification accuracy of 87.50% (AUC: 0.917, sensitivity: 95%, specificity: 75%, f1-score: 90.48%) using 8-bit LBP variant-I. 10-fold cross-validated accuracy of all four sets range from 65.62% to 87.50% using random forest classifier and mean-AUC range from 0.646 to 0.917.

Random forest is one of the most heated machine learning tools in a wide range of industrial scenarios. Recently, federated learning enables efficient distributed machine learning without direct revealing of private participant data. In this paper, we present a novel framework of federated random forest (RevFRF), and further emphatically discuss the participant revocation problem of federated learning based on RevFRF. Specifically, RevFRF first introduces a suite of homomorphic encryption based secure protocols to implement federated random forest (RF). The protocols cover the whole lifecycle of an RF model, including construction, prediction, and participant revocation. Then, referring to the practical application scenarios of RevFRF, the existing federated learning frameworks ignore a fact that even every participant in federated learning cannot maintain the cooperation with others forever. In company-level cooperation, allowing the remaining companies to use a trained model that contains the memories from an off-lying company potentially leads to a significant conflict of interest. Therefore, we propose the revocable federated learning concept and illustrate how RevFRF implements participant revocation in applications.

With the rise in popularity and its open system architecture, Android has become vulnerable to malicious attacks. There are several malware detection approaches available to fortify the Android operating system from such attacks. These malware detectors classify target applications based on the patterns found in the features present in the Android applications. As the analytics data continues to grow, it negatively impacts the Android defense mechanism. A large number of irrelevant features has become the performance bottleneck of the detection mechanism. This paper presents a multi-tiered feature selection model, which can discover relevant and significant features for improving the accuracy of malware detection approaches. The proposed method applies five machine learning classification techniques to the selected feature set. This work presents the Optimal Static Feature Set (OSFS) and, Most Important Features (MIFs) discovered with each machine learning approach. Rigorous testing and analysis show that Random Forest classification achieves the highest Accuracy rate of 96.28%.

There is an increasing interest in the study of detecting psychological stress from the social media like Twitter. However, Twitter has a limitation on the number of characters used per tweet, resulting in data sparsity. Many techniques were proposed to detect stress at the tweet level, but most of them failed to leverage the text content to reduce the impact of data sparsity. In this work, two solutions are proposed to further leverage the text content for the tweet level stress detection. First, a new feature, Sarcasm_Level, is computed to indicate the sarcasm present in the tweet’s content and its influence in detecting stress. Second, a novel neighborhood tweet-based stress detection method is developed, which is a logistic regression-based approach that integrates the content of previous tweets, also known as neighborhood tweets. Experimental results reveal that the proposed model has significantly better performance in detecting stress since it incorporates the information from neighborhood tweets. Also, with the inclusion of the new feature, the proposed model outperformed other well-known machine learning models like support vector machines, random forests and logistic regression with better accuracy and F1-score.

The economic boom over the recent past and the quest to further develop has made several nation states the business hubs in their regions. Along with the investments, there has been growth in the number of property sales. Social media has become convenient platform of choice for advertising property sales after the introduction of Web 2.0. This article utilizes social media platforms like Facebook to scrape data from user groups advertising properties and then using data mining techniques and approaches to determine true valuation of properties. This methodology is based on set attributes in the urban areas by looking at the property sales of the recent past within the same area. This enables investors interested in these properties and provides a fair idea of price of properties based on the key attributes associated with the respective property.

Cow behavior recognition systems support the assessment of cows’ condition by providing their behavior information. Accelerometers are particularly suited for a non-invasive solution for the development of these monitoring systems. They are cheap and simple in setting up and providing high-performance recognition when using machine learning algorithms. The activity complexity of animals brings challenges in real context applications because different behaviors may have similar acceleration data. The reason is that they contain similar gestures, for example, feeding and standing. In our previous work, we proposed a cows’ behavior classifier based on leg-mounted acceleration data. The distinguishing of similar behaviors, such as feeding and standing, is limited by the data. This study presents a new efficient cow behavior recognition system based on combining leg-mounted and collar-mounted accelerometers. Significantly, the acceleration data from these two sensors were synchronized. Therefore, we can substantially expand the amount of information for classification purposes. Our approach identifies four cow behaviors: walking, feeding, lying, and standing. Random Forest algorithm with our extracted features (root mean square, standard deviation, and mean) and 16-second data window (a sample/second) offer excellent performance when identifying all concerning behaviors: feeding (0.914 accuracy, 0.884 sensitivity, 0.956 positive predictive value), lying (0.998, 0.996, 1), standing (0.88, 0.928, 0.842), and walking (0.998, 0.996, 0.998). These performances are better than other existing works, especially in our experiments with free-grazing cows.

Music Information Retrieval (MIR) is the task of extracting high-level information, such as genre, artist or instrumentation from music. Genre classification is an important and rapidly evolving research area of MIR. To date, only a small amount of research work has been done on the automatic genre classification of Nigerian songs. Hence, this study presents a new music dataset, namely the ORIN dataset, consisting of only Nigerian songs. The study dataset contains 478 Nigerian traditional songs from five genres: fuji, juju, highlife, waka and apala. The timbral texture and tempo features were mined from 30-second segments of each song using the Librosa Python library. For genre classification, the ORIN datasets was trained on 4 different classifiers- k-Nearest Neighbhour, Support Vector Machine, eXtreme Gradient Boosting (XGBoost) and Random Forest- with 85–15 train-test splits. The results obtained for the five different genres indicates that XGBoost classifier is a better model, having the highest accuracy of 81.94% and recall of 84.57%. This study uses the global mean (Tree SHAP) method to determine feature importance and impact on the classification model. Further analysis on the individual genres found some nearness in the timbral properties between some of the genres. This analysis was confirmed by human observation.

Spatiotemporal models that incorporate information about the relationships between traffic flows over space and time have recently become the mainstream of urban traffic forecasting. Many algorithms have been developed to learn the structure of spatiotemporal relationships - the travel time-based, cross-correlation, graphical lasso, mutual information, and random forest methods, among several others. In this paper, we address two issues of spatiotemporal relationship learning. First, we suggest the application of ensemble techniques (ranking- and voting-based) that combine the advantages of individual algorithms and allow more robust structures to be obtained for learning spatiotemporal relationships. Second, we propose strategies that enable discovery of their dynamics and provide extensive information on traffic flows for the responsive learning of spatiotemporal relationships. In addition to the widely used forecasting performance (estimated for the spatially regularised vector autoregression model, the spatiotemporal k-nearest neighbour algorithm, and the high-order graph convolutional artificial neural network), we compare the estimated spatiotemporal relationship structures using their complexity and stability metrics and demonstrate the importance of these characteristics. A multitude of numerical experiments using a large citywide traffic data set are conducted, and the results demonstrate the preferences of ensemble learning and the strengths of the proposed learning strategies.

A record of spatial and temporal parking occupancy is critical to optimize on-street parking resources and to develop effective parking policies. Such data are often obtained through advanced and costly occupancy monitoring technologies. Moreover, it is usually challenging to integrate bay-level occupancies and parking data from other systems. Accurate occupancy-payment data are required for a wide range of analytical and practical purposes, including but not limited to investigating payment behavior, estimating and forecasting occupancy, and evaluating the efficiency and effectiveness of enforcement policies. This study proposes a metaheuristic optimization algorithm to integrate snapshots of bay-level parking occupancy, captured using simple cameras, with transactions from a conventional parking payment management system. The resulting integrated data were used to develop, calibrate and validate a parking occupancy estimation method utilizing parking payment data only. Details of the design, implementation, and validation of the proposed algorithm and modelling technique are provided. Logistic regression analysis was used to tune parameters of the data integration algorithm. Deep learning, gradient boosting and random forests were used to develop a model of parking occupancy. Evaluation of the algorithm indicated an accuracy of 76% of correct data integration; that is, individual bay occupancies integrated with the correct corresponding payment transactions. The best occupancy estimation model also showed a very high accuracy, with an R² above 94% and a root mean square error (RMSE) of 1.2 (occupied bays), when tested with a random sample from the integrated data.

Diagnosing the fault as early as possible is significant to guarantee the safety and reliability of the high-speed train. Incipient fault always makes the monitored signals deviate from their normal values, which may lead to serious consequences gradually. Due to the obscure early stage symptoms, incipient faults are difficult to detect. This article develops a stacked generalization (stacking)-based incipient fault diagnosis scheme for the traction system of high-speed trains. To extract the fault feature from the faulty data signals, which are similar to the normal ones, the extreme gradient boosting (XGBoost), random forest (RF), extra trees (ET), and light gradient boosting machine (LightGBM) are chosen as the base estimators in the first layer of the stacking. Then, the logistic regression (LR) is taken as the meta estimator in the second layer to integrate the results from the base estimators for fault classification. Thanks to the generalization ability of stacking, the incipient fault diagnosis performance of the proposed stacking-based method is better than that of the single model (XGBoost, RF, ET, and LightGBM), although they can be used to detect the incipient faults, separately. Moreover, to find out the optimal hyperparameters of the base estimators, a swarm intelligent optimization algorithm, pigeon-inspired optimization (PIO), is employed. The proposed method is tested on a semiphysical platform of the CRH2 traction system in CRRC Zhuzhou Locomotive Company Ltd. The results show that the fault diagnosis rate of the proposed scheme is over 96%.

Studies have estimated that, in industrial settings, developers spend between 30% and 90% of their time fixing bugs. As such, tools that assist in identifying the location of bugs provide value by reducing debugging costs. One such tool is BugLocator. This study initially aimed to determine if developers working on the Adobe Analytics product could use BugLocator. The initial results show that BugLocator achieves a similar accuracy on 5 of 7 Adobe Analytics repositories and on open-source projects. However, these results do not meet the minimum applicability requirement deemed necessary by Adobe Analytics developers prior to possible adoption. Thus, we consequently examine how BugLocator can achieve the targeted accuracy with two extensions: (1) adding more data corpora, and (2) massively composing individual rankers consisting of augmented BugLocator instances trained on various combinations of corpora and parameter configurations with a Random Forest model. We refer to our final extension as Legion. On average, applying Legion to Adobe Analytics repositories results in at least one buggy file ranked in the top-10 recommendations 76.8% of the time for customer-reported bugs across all 7 repositories. This represents a substantial improvement over BugLocator of 36.4%, and satisfies the minimum applicability requirement. Additionally, our extensions boost Mean Average Precision by 107.7%, Mean Reciprocal Rank by 86.1%, Top 1 by 143.4% and Top 5 by 58.1%.

To improve the thrust performance of permanent magnet synchronous linear motor (PMSLM), novel deep adaptive ridge regression with embedded analytical mapping function (EAMF-RR) is proposed to calculate the thrust performance of PMSLM quickly and accurately, and combined with the EAMF-RR, a whale optimization algorithm (WOA) is introduced to optimize the PMSLM structure. First, the PMSLMs thrust is analyzed by the analytical modeling analysis (AMA) to determine the variation range of structural design parameters. Based on the variation range, a finite-element sample database is established for the EAMF-RR. Then, embedded analytical mapping functions (EAMF) derived from AMA, k-nearest neighbor algorithm, and WOA are integrated into ridge regression to establish deep adaptive EAMF-RR model. Comparative simulation experiments conducted by using another two machine learning methods, i.e., extreme learning machine and weighted random forest, confirm the superiority of the EAMF-RR. Finally, WOA is reused to optimize the thrust performance of PMSLM, and prototype experiments prove the effectiveness of the proposed optimization method.

This article introduces ILA4: A new algorithm designed to handle datasets with missing values. ILA4 is inspired by a series of ILA algorithms which also handle missing data with further enhancements. ILA4 is applied to datasets with varying completeness and also compared to other, known approaches for handling datasets with missing values. In the majority of cases, ILA4 produced favorable performance that is on a par with many established approaches for treating missing values including algorithms that are based on the Most Common Value (MCV), the Most Common Value Restricted to a Concept (MCVRC), and those that utilize the Delete strategy. ILA4 was also compared with three known algorithms namely: Logistic Regression, Naive Bayes, and Random Forest; the accuracy obtained by ILA4 is comparable or better than the best results obtained from these three algorithms.

Mitigating label noise is a crucial problem in classification. Noise filtering is an effective method of dealing with label noise which does not need to estimate the noise rate or rely on any loss function. However, most filtering methods focus mainly on binary classification, leaving the more difficult counterpart problem of multiclass classification relatively unexplored. To remedy this deficit, we present a definition for label noise in a multiclass setting and propose a general framework for a novel label noise filtering learning method for multiclass classification. Two examples of noise filtering methods for multiclass classification, multiclass complete random forest (mCRF) and multiclass relative density, are derived from their binary counterparts using our proposed framework. In addition, to optimize the NI_threshold hyperparameter in mCRF, we propose two new optimization methods: a new voting cross-validation method and an adaptive method that employs a 2-means clustering algorithm. Furthermore, we incorporate SMOTE into our label noise filtering learning framework to handle the ubiquitous problem of imbalanced data in multiclass classification. We report experiments on both synthetic data sets and UCI benchmarks to demonstrate our proposed methods are highly robust to label noise in comparison with state-of-the-art baselines. All code and data results are available at https://github.com/syxiaa/Multiclass-Label-Noise-Filtering-Learning.

This paper proposes a hybrid machine learning method for the reliability evaluation of integrated power-gas systems (IPGS) under the uncertain component failure probability distributions. The Random Forest (RF) method is designed to select important features to solve the insufficient quantity of data and the curse of dimensionality problems. The Extreme Gradient Boosting (XGBoost) regression algorithm is developed to quantify the relationship between the uncertain parameters and reliability metrics. Moreover, a ten-fold cross-validation method is employed to further improve the accuracy of the regression model. Simulation results on three test systems show that the proposed method can achieve high accuracy for the reliability evaluation.

Implementing a proper clustering algorithm and a high accuracy classifier for applying on electricity smart meter data is the first stage in analyzing and managing electricity consumption. In this paper, Random Forest (RF) classifier optimized by Artificial Bee Colony (ABC) which is called Artificial Bee Colony-based Random Forest (ABC-RF) is proposed. Also, in order to determine the representative load curves, the Convex Clustering (CC) is used. The solution paths generated by convex clustering show relationships among clusters that were hidden by static methods such as k-means clustering. To validate the proposed method, a case study that includes a real dataset of residential smart meters is implemented. The results evidence that the proposed ABC-RF method provides a higher accuracy if compared to other classification methods.

Researchers have created several expert systems over the years to predict heart disease early and assist cardiologists to enhance the diagnosis process. We present a diagnostic system in this paper that utilizes an optimized XGBoost (Extreme Gradient Boosting) classifier to predict heart disease. Proper hyper-parameter tuning is essential for any classifier’s successful application. To optimize the hyper-parameters of XGBoost, we used Bayesian optimization, which is a very efficient method for hyper-parameter optimization. We also used One-Hot (OH) encoding technique to encode categorical features in the dataset to improve prediction accuracy. The efficacy of the proposed model is evaluated on Cleveland heart disease dataset and compared it with Random Forest (RF) and Extra Tree (ET) classifiers. Five different evaluation metrics: accuracy, sensitivity, specificity, F1-score, and AUC (area under the curve) of ROC charts were used for performance evaluation. The experimental results showed its validity and efficacy in the prediction of heart disease. In addition, proposed model displays better performance compared to the previously suggested models. Moreover, our proposed method reaches the high prediction accuracy of 91.8%. Our results indicate that the proposed method could be used reliably to predict heart disease in the clinic.

N2-methylguanosine is a post-transcriptional modification of RNA that is found in eukaryotes and archaea. The biological function of m2G modification discovered so far is to control and stabilize the three-dimensional structure of tRNA and the dynamic barrier of reverse transcription. To discover additional biological functions of m2G, it is necessary to develop time-saving and labor-saving calculation tools to identify m2G. In this paper, based on hybrid features and a random forest, a novel predictor, RFhy-m2G, was developed to identify the m2G modification sites for three species. The hybrid feature used by the predictor is used to fuse the three features of ENAC, PseDNC, and NPPS. These three features include primary sequence derivation properties, physicochemical properties, and position-specific properties. Since there are redundant features in hybrid features, MRMD2.0 is used for optimal feature selection. Through feature analysis, it is found that the optimal hybrid features obtained still contain three kinds of properties, and the hybrid features can more accurately identify m2G modification sites and improve prediction performance. Based on five-fold cross-validation and independent testing to evaluate the prediction model, the accuracies obtained were 0.9982 and 0.9417, respectively. The robustness of the predictor is demonstrated by comparisons with other predictors.

In the process industry, it is essential to establish a data-driven soft sensor to predict the key variable that is difficult to online measure directly. The accuracy performance of data-driven soft sensors relies heavily on data. Unfortunately, it is hard to acquire sufficient and informative data from the samples with limited number, which is called as the small sample problem. For handling the small sample problem, it is a good solution to generating virtual samples according to the distribution of original data. This paper proposes an enhanced method of virtual sample generation utilizing manifold features to develop soft sensors using small data. First, T-Distribution Stochastic Neighbor Embedding (t-SNE) is utilized to extract the features of input data. The main idea of generating virtual samples is to use the interpolation algorithm to obtain virtual t-SNE input features and then the random forest algorithm is utilized to get the virtual outputs using virtual t-SNE input features. Finally, virtual samples using the proposed t-SNE based virtual sample generation (t-SNE-VSG) can be achieved. For the sake of confirming the effectiveness and feasibility of the presented t-SNE-VSG, a standard data set is first used. What is more, a small data set from an actual industrial process of Purified Terephthalic Acid is used to establish a soft sensor model. The results from simulations show that the accuracy performance of the soft sensor established with small data can be effectively improved by adding the virtual samples generated by t-SNE-VSG. In addition, t-SNE-VSG achieves superior accuracy to state-of-the-art virtual sample generation methods.

The study involves an object-based segmentation method to extract feature changes in tropical rainforest cover using Landsat image and airborne LiDAR (ALS). Disturbance event that are represents the changes are examined by the classification of multisensor data; that is a highly accurate ALS with different resolutions of multispectral Landsat image. Disturbance Index (DI) derived from Tasseled Cap Transformation, Normalized Difference Vegetation Index (NDVI), and the ALS height are the variables for object-based segmentation process. The classification is categorized into two classes; disturbed and non-disturbed forest cover using Nearest Neighbor (NN), Random Forest (RF) and Support Vector Machine (SVM). The overall accuracy ranging from 88% to 96% and kappa ranging from 0.79 to 0.91. Mcnemar’s test p-value (

The integration of distributed renewable energy and the implementation of the demand response complicate the change patterns of load profiles and present great uncertainties. Probabilistic load forecasting, an effective method for capturing the future load uncertainty, has been a hot issue. This article proposes a two-stage bootstrap sampling method for probabilistic load forecasting. In the first stage, alpha-bootstrap, which is a modification of traditional bootstrap methods, is applied to characterize the uncertainties from multiple forecasting models; in the second stage, the residual bootstrap method is used to formulate the regression errors. Finally, the probabilistic load forecasts can be obtained by integrating the uncertainties in both stages. Various off-the-shelf point load forecasting methods such as random forest (RF) and gradient boosting regression tree (GBRT) can be integrated into the proposed framework. We illustrate the effectiveness of our proposed method and superiority over direct quantile regression methods such as quantile RF and quantile GBRT using the case studies on open load datasets of eight zones in ISO New-England.

To evaluate the performance of the classic machine learning algorithms and the effectiveness of various features, the iterative algorithms (i.e., support vector machine (SVM), and least-squares SVM (LS-SVM)) and non-iterative algorithms (i.e., random forest (RF) and naive bayes (NB)) for six feature schemes were performed to classify the ECG recordings. The ECG recordings were initially filtered with a 0.1 Hz − 12 Hz band pass filter. Then 80 features, including 48 time domain, 18 frequency domain, 12 time–frequency and two principle component analysis (PCA) features, were extracted to construct six feature schemes. The RF, SVM, LS-SVM and NB were employed to discern a binary-classification task (i.e., normal and AF ECG recordings) and a tri- classification task (i.e., the normal, AF and ST change ECG recordings) for the six feature schemes. The results revealed that time domain, frequency domain features and PCA features can provide relatively reliable feature combinations to the RF and SVM. In addition, the RF yielded the highest F1-scores (0.8908 and 0.7535) for the binary-classification task and the tri-classification task than the SVM, LS-SVM and NB.

The consumer’s short term load forecasting plays an essential role in microgrid energy distribution. However, the load forecasting at consumer level is more challenging than at substation level due to high volatility and uncertainty in energy consumption. In literature, many machine learning-based forecasting models have been explored. However, there is a need of developing a robust and accurate model to handle highly inconsistent energy consumption. In this article, we propose a robust and accurate model for consumer’s short term load forecasting, which uses feasible techniques such as random forest, support vector regressor, and long short term memory as base predictors to handle varying traits of energy consumption. For the final decision-making on forecasting result of these predictors, it assigns weights to each predictor dynamically as per the forecasting efficacy. The proposed model is tested on different consumer’s varying traits of energy consumption. The experimental results show that the proposed model achieves forecasting error reduction by 3.46 and 2.53 in terms of average RMSE and MAE, respectively, in comparison with the existing models. It is robust and accurate even in presence of highly volatile and uncertain load patterns, and thus, it can better fit for microgrid energy management.

This article proposes an adaptive state of health (SOH) estimation method for lithium-ion batteries using machine learning. Practical problems with feature extraction, cell inconsistency, and online implementability are specifically solved using a proposed individualized estimation scheme blending offline model migration with online ensemble learning. First, based on the data of pseudo-open-circuit voltage measured over the battery lifespan, a systematic comparison of different incremental capacity features is conducted to identify a suitable SOH indicator. Next, a pool of candidate models, composed of slope-bias correction (SBC) and radial basis function neural networks (RBFNNs), are trained offline. For online operation, the prediction errors due to cell inconsistency in the target new cell are then mitigated by a proposed modified random forest regression (mRFR) based ensemble learning process with high adaptability. The results show that compared to prevailing methods, the proposed SBC-RBFNN-mRFR-based scheme can achieve considerably improved SOH estimation accuracy (15%) with only a small amount of early-age data and online measurements are needed for practical operation. Furthermore, the applicability of the proposed SBC-RBFNN-mRFR algorithms to real-world operation is validated using measured data from electric vehicles, and it is shown that a 38% improvement in estimation accuracy can be achieved.

A large number of countries depend on agriculture. However, the biggest challenge for farmers is the plant-related diseases. The most observed part of the plant is leaf but the segmentation of a diseased lesion leaf image suffers from uneven illumination and cluttered complex natural environment problems. To overcome these problems, the proposed methodology is based on the fusion of color balancing and superpixel. First, the input image transforms into a color-balanced image to eliminate the effects of uneven illumination. Second, compact regions are created from the transformed image using superpixel operation and the empirically derived threshold is applied on Histogram of Gradients (HOG) and color channels of superpixel to separate the unwanted background from leaf image. K-means clustering is applied to find a diseased infected image. An extended form of HOG, Pyramid of HOG(PHOG) with Gray Level Co-occurrence Matrix (GLCM) features are used to represent a diseased infected part. Finally, different classifiers are compared and Random Forest (RF) is selected to classify diseases. To assess the robustness of the proposed method, an experiment is conducted to test its accuracy effectively on natural images acquired from various farms, downloaded images from the Internet and Plant-Village color dataset. Results indicate that the proposed method achieved an accuracy of 93.12% on a combined dataset using cross-validation to show that method is effective to classify diseases in the presence of a cluttered background. The proposed system is compared with ALDD method (Dhaygude and Kumbhar, 2013), MVBPDR (Habib et al., 2018), AIMSD (Araujo and Peixoto, 2019) and CCF (Ma et al., 2017). The novelty of the proposed system lies in the fact that the proposed segmentation is effective for all the varied datasets and it is evaluated by True Positive Rate (TPR), False Positive Rate (FPR), False Negative Rate (FNR), Positive Prediction Value (PPV), False Discovery Rate (FDR), Accuracy and F1 score.

Decision tree is a widely used non-parametric technique in machine learning, data mining and pattern recognition. It is simple to understand and interpret, however it faces challenges such as handling higher dimensional and class imbalanced datasets, over-fitting and instability. To overcome some of these issues, vertical partitioning approaches like serial partitioning, theme based partitioning are used in the literature. A vertical partitioning approach divides the feature set into subsets of features (blocks) and makes use of these subsets for subsequent tasks. In this work, we use the ideas of music rhythm tree to propose a novel vertical partitioning technique. It orders the features based on the average correlation strength of the features before partitioning the feature set. The proposed method is proved to be superior by showing an average of 13.8 % , 6 % , 9.8 % , 19.7 % , 9.4 % , and 29.4 % higher classification accuracy over C4.5, Random Forest, Bagging, Adaboost, an ensemble technique and a vertical partitioning technique respectively. Our empirical results on 15 datasets demonstrate that the proposed vertical partitioning method is more stable and better in handling class-imbalanced data. Finally, some popular statistical tests are conducted to validate the statistical significance of the results of the proposed method.

An accurate prediction of ship fuel consumption is critical for speed, trim, and voyage optimisation etc. While previous studies have focused on predicting ship fuel consumption with respect to a variety of factors, research on the impact of environmental factors on fuel consumption has been lacking. In addition, although recent research efforts have widely focused on machine learning methods to predict fuel consumption, studies on hyperparameter values that are suitable for these prediction models are limited. To compensate for this deficiency in existing literature, an adaptive hyperparameter tuning method is proposed, and the effects of maritime environmental factors on fuel consumption are taken into account. Through experimentation, the proposed adaptive hyperparameter tuning method was validated via artificial neural network (ANN), support vector regression (SVR), random forest (RF), and least absolute shrinkage and selection operator (Lasso). The hyperparameter tuning proportionally increased the amplitudes of the coefficients of determination (R2) of these algorithms. The increase of the amplitude demonstrated the following trend, in the order of the largest increase to the lowest increase: ANN, Lasso, SVM, and RF. The rates of increase were between 0.0773% and 2.1653%. Furthermore, after the environmental factors were considered, the prediction accuracies of the ANN and Lasso increased; however, the opposite was observed for the SVR and RF. As such, we confirmed that the use of Bayesian optimisation for hyperparameter tuning can effectively improve the fuel consumption prediction accuracy, and our proposed model can therefore serve as a significant reference for calculating fuel consumption.

Machine learning and artificial intelligence approaches have rapidly gained popularity for use in many subsurface energy applications. They are seen as novel methods that may enhance existing capabilities, providing for improved efficiency in exploration and production operations. Furthermore, their integration into reservoir management workflows may shape the future landscape of the energy industry. This study implements a framework that generates predictive models using multiple machine learning classification-based algorithms which can identify specific stratigraphic units (i.e., formations) as a function of total vertical depth and spatial positioning. The framework is applied in a case study to 13 specific formations of interest (Upper Spraberry through Atoka / Morrow reservoirs) in the Midland Basin, West Texas, United States; a prominent hydrocarbon producing sub-basin of the larger Permian Basin. The study dataset consists of over 275,000 records and includes data fields like formation identifier, true vertical depth (in feet) of formations observed, and latitude and longitude coordinates (in decimal degrees). A subset of 134,374 data records were relevant to the 13 distinct formations of interest and were extracted and used for machine learning model training, validation, and testing. Four supervised learning approaches including random forest (RF), gradient boosting (GB), support vector machine (SVM), and multilayer perceptron neural network (MLP) were evaluated and their prediction accuracy compared. The best performing model was ultimately built on the RF algorithm and is capable of an overall prediction accuracy of 93 percent on holdout data. The RF-based model demonstrated high prediction accuracy for major oil and gas producing zones including the San Andres, Upper Spraberry, Lower Spraberry, Clearfork, and Wolfcamp at 98, 94, 89, 94, and 94 percent respectively. Overall, the resulting data-driven model provides a robust, cost-effective approach which can complement contemporary reservoir management approaches for multiple subsurface energy applications.

This paper focuses on automatic Cholangiocarcinoma (CC) diagnosis from microscopic hyperspectral (HSI) pathological dataset with deep learning method. The first benchmark based on the microscopic hyperspectral pathological images is set up. Particularly, 880 scenes of multidimensional hyperspectral Cholangiocarcinoma images are collected and manually labeled each pixel as either tumor or non-tumor for supervised learning. Moreover, each scene from the slide is given a binary label indicating whether it is from a patient or a normal person. Different from traditional RGB images, the HSI acquires pixels in multiple spectral intervals, which is added as an extension on the channel dimension of 3-channel RGB image. This work aims at fully exploiting the spatial-spectral HSI data through a deep Convolution Neural Network (CNN). The whole scene is first divided into several patches. Then they are fed into CNN for the tumor/non-tumor binary prediction and the tumor area regression. The further diagnosis on the scene is made by random forest based on the features from patch prediction. Experiments show that HSI provides a more accurate result than RGB image. Moreover, a spectral interval convolution and normalization scheme are proposed for further mining the spectral information in HSI, which demonstrates the effectiveness of the spatial-spectral data for CC diagnosis.

Alzheimer’s disease (AD) is the most common type of neurological disorder that leads to the brain’s cell death overtime. It is one of the major important causes of memory loss and cognitive decline in elderly subjects around the globe. Early detection and streamlining of diagnostic practices are the prime domains of the interest to the healthcare community. Machine learning (ML) algorithms and numerous multivariate data exploratory tools have been extensively used in the field of AD research. The primary purpose of this study is to present an automated classification system to retrieve information patterns. We proposed a five-stage ML pipeline, where each stage was further categorized in different sub-levels. The study relied on the Open Access Series of Imaging Studies (OASIS) database of MRI (Magnetic Resonance Imaging) brain images for the analysis. The dataset comprised of 343 MRI sessions involving 150 subjects. Three different scores namely, MMSE (Mini-Mental State Examination), CDR (Clinical Dementia Rating), and ASF (Atlas Scaling Factor) were used in the analysis. The proposed ML pipeline constitutes a classifier system along with data transformation and feature selection techniques that have been embedded inside an experimental and data analysis design. Performance metrics for Random Forest (RF) classifier showed the highest output in the classification accuracy.

Traffic prediction is a spatiotemporal predictive task that plays an essential role in intelligent transportation systems. Today, graph convolutional neural networks (GCNNs) have become the prevailing models in the traffic prediction literature since they excel at extracting spatial correlations. In this work, we classify the components of successful GCNN prediction models and analyze the effects of matrix factorization, attention mechanism, and weight sharing on their performance. Furthermore, we compare these variations against random forests, a traditional regression method that predates GCNNs by over 15 years. We evaluated these methods using simulated data of two regions in Toronto as well as real-world sensor data from selected California highways. We found that incorporating matrix factorization, attention, and location-specific model weights either individually or collectively into GCNNs can result in a better overall performance. Moreover, although random forest regression is a less compact model, it matches or exceeds the performance of all variations of GCNNs in our experiments. This suggests that the current graph convolutional methods may not be the best approach to traffic prediction and there is still room for improvement. Finally, our findings also suggest that for future research on GCNN for traffic prediction to be credible, researchers must include performance comparison to random forests.

Introduction: Intracranial aneurysms (IAs) are a common vascular pathology and are associated with a risk of rupture, which is often fatal. Aneurysm growth is considered a surrogate of rupture risk; therefore, the study aimed to develop and evaluate prediction models of future artificial intelligence (AI) growth based on baseline aneurysm morphology as a computer-aided treatment decision support.Materials and methods: Follow-up CT angiography (CTA) and magnetic resonance angiography (MRA) angiograms of 39 patients with 44 IAs were classified by an expert as growing and stable (25/19). From the angiograms vascular surface meshes were extracted and the aneurysm shape was characterized by established morphologic features and novel deep shape features. The features corresponding to the baseline aneurysms were used to predict future aneurysm growth using univariate thresholding, multivariate random forest and multi-layer perceptron (MLP) learning, and deep shape learning based on the PointNet++ model.Results: The proposed deep shape feature learning method achieved an accuracy of 0.82 (sensitivity = 0.96, specificity = 0.63), while the multivariate learning and univariate thresholding methods were inferior with an accuracy of up to 0.68 and 0.63, respectively.Conclusion: High-performing classification of future growing IAs renders the proposed deep shape features learning approach as the key enabling tool to manage rupture risk in the “no treatment” paradigm of patient follow-up imaging.

BACKGROUND: Breast cancer is a major disease causing panic among women worldwide. Since gene mutations are the root cause for cancer development, analyzing gene expressions can give more insights into various phenotype of cancer treatments. Breast Cancer subtype prediction from gene expression data can provide more information for cancer treatment decisions. OBJECTIVE: Gene expressions are complex for analysis due to its high dimensional nature. Machine learning algorithms such as k-Nearest Neighbors, Support Vector Machine (SVM) and Random Forest are used with selection of features for prediction of breast cancer subtypes. Prediction accuracy of the existing methods are affected due to high dimensional nature of gene expressions. The objective of the work is to propose an efficient algorithm for the prediction of breast cancer subtypes from gene expression. METHODS: For subtype prediction, a novel Hubness Weighted Support Vector machine algorithm (HWSVM) using bad hubness score as a weight measure to handle the outliers in the data has been proposed. Based on the various subtypes, features are projected into seven different feature sets and Ensemble based Hubness Aware Weighted Support Vector Machine (HWSVMEns) is implemented for breast cancer subtype prediction. RESULTS: The proposed algorithms have been compared with the classical SVM and other traditional algorithms such as Random Forest, k-Nearest Neighbor algorithms and also with various gene selection methods. CONCLUSIONS: Experimental results show that the proposed HWSVM outperforms other algorithms in terms of accuracy, precision, recall and F1 score due to the hubness weightage scheme and the ensemble approach. The experiments have shown an average accuracy of 92% across various gene expression datasets.

This paper proposes a cascading failure simulation (CFS) method and a hybrid machine learning method for vulnerability analysis of integrated power-gas systems (IPGSs). The CFS method is designed to study the propagating process of cascading failures between the two systems, generating data for machine learning with initial states randomly sampled. The proposed method considers generator and gas well ramping, transmission line and gas pipeline tripping, island issue handling and load shedding strategies. Then, a hybrid machine learning model with a combined random forest (RF) classification and regression algorithms is proposed to investigate the impact of random initial states on the vulnerability metrics of IPGSs. Extensive case studies are carried out on three test IPGSs to verify the proposed models and algorithms. Simulation results show that the proposed models and algorithms can achieve high accuracy for the vulnerability analysis of IPGSs.

Women worldwide are afflicted by breast cancer, a widely occuring health issue that causes a large number of deaths. This paper aims to review and present several approaches to identify breast cancers using ML algorithms and Biosensors. The objective is to investigate the application of multiple algorithms based on Machine Learning approach and biosensors for early breast cancer detection. Biosensors and machine learning are needed to identify cancers based on microscopic images, that is why automation is needed. ML aims to make computers capable of self-learning. Rather than relying on explicit pre-programmed rules and models, it is based on identifying patterns in observed data and building models to predict outcomes. We have compared and analyzed various types of algorithms like fuzzy ELM-RBF, SVM, SVR, RVM, Naive Bayes, K-NN, DT, ANN, BPNN and Random forest over varieties of databases like images digitized from fine needle aspirations (FNAs) of breast masses, Scanned film mammography, Breast infrared images, MR Images, Data collected by using the blood analyses, and Histopathology image samples. The result is compared on performance metric elements like accuracy, precision and recall. Further, we have used Biosensors to determine the presence of a specific biological analyte by transforming the cellular constituents of proteins, DNA, or RNA into electrical signals that can be detected and analyzed. Here, we have compared detection of different types of analyte like HER2, miRNA 21, miRNA 155, MCF-7 cells, DNA, BRCA1, BRCA2, human tears and saliva using different types of biosensors like FET, Electrochemical, Sandwich electrochemical, etc. Several biosensors use a different type of specification which is also discussed. The result of which is analysed on the basis of detection limit, linear ranges and response time. Different studies and related articles were ultimately reviewed and analyzed systematically and the ones published from 2010 to 2021 were considered. Biosensors and machine learning both have the potential to detect breast cancer in a fast and effective way.